My Adventures with Narrative Engines

I've always loved games with compelling stories. From the classic LucasArts point & click adventures to modern gems like The Forgotten City, Disco Elysium, Pentiment, and Telltale games, many titles have inspired me.

While working on my latest project, I wondered: how would I implement a story, with dialogues and potentially complex narratives?

Initially, it seemed like stories are just a series of “if” statements: "if the player gives the red herring to the guard troll, display that line of dialogue". But surely, you don’t want to put all the lines of dialogues and conditions in your source code, do you?

In this blog post, I'm sharing my experience and my discoveries.

Disclaimer: I’m not a professional game developer. I’m just sharing my thoughts.

A Proof of Concept

Fundamentally, what I want is a text containing a set of choices. Like in Choose Your Own Adventure (CYOA) books, each choice can simply reference a different section.

It’s easy to put it in a JSON file:

{
  "sections": {
    "1": {
      "text": "You are in a dark room...",
      "choices": [
        {"text": "Look around", "goto": "2"},
        {"text": "Go back to sleep", "goto": "3"}
      ]
    },
    "2": {
      "text": "You find a door...",
      …

The player makes choices, gets to a new section, makes new choices, etc. This basic mechanism can actually be enough to tell a story.

The story graph might be acyclic, so that the story always progresses like in a book. The graph could also contain cycles, in which case it can allow the player to go back and forth between multiple locations, but it also leads to text repetition. So for example, a character might always repeat the same line, “Morning, nice day for fishin’ ain't it?

Despite the simplicity, managing large stories in JSON can quickly become cumbersome. We need to work a bit more.

A New Language

Instead of directly using JSON, we can look for a better alternative:

  • Some people will want to build an editor. Each section can go in its own box, and choices are represented visually with arrows between boxes.

  • Some other people will prefer using a plain text file, which lets them use their favorite text editor and all the existing tooling.

I personally like working with languages. Let's imagine a very simple markup language. For example:

=== section1 ===

You enter a bar with pirates.

* Talk to the dog -> section2
* Talk to the important-looking pirates -> section3
* Enter the kitchen -> section4

=== section2 ===

Grrrr…

It is very simple to parse this kind of file. Using this small language (or a custom editor), we can significantly streamline the development process. It will also help collaborating with a writer.

Even though we can already tell stories with this, it will feel very limited. How would you pick up an object and use it later?

State and Conditions

So a clear limitation is the absence of state management. The game doesn’t remember what the player did (in theory, you could duplicate the paths in the graph to remember that, but that’s stupid).

CYOA books solve this problem by asking you to take notes during the game, and spell out the conditions. With a computer, we can instead evaluate the condition automatically. So let’s add conditions to the JSON file:

    "choices": [
      {"text": "Use the key", "goto": "2", "conditions": ["has_key"]},
      {"text": "Kick the door", "goto": "3"}
    ]

Similarly, we can add an actions field to pick up an object (or set any other kind of boolean value). Using the text mini-language, we can imagine a syntax like this:

What do you do?

* {has_key} Use the key -> section2
* Kick the door -> section3

With actions and conditions, we can make more interesting stories. Depending on the path you took, you might have a different inventory. Your actions can have an impact on the virtual world, which might influence the ending of the story and add replay value.

That’s great, but if you’re an ambitious storyteller, you might think of dozens of missing features: Could we randomize some events? Could we have integer variables, for example for money management, as well as integer comparisons to know if we can buy an item? Can we remove the choices once they have been used? Oh, and dialogues can be tedious to write, if we have to create a new section for each sentence.

None of this seems difficult to do, but it can be a lot of work to implement and polish everything. And we might wonder if we are reinventing the wheel…

Reusing the Wheel

So I used my favorite search engine to look for a wheel. I’ve found a few, and I saw that Amazon offers good discounts.

But then something caught my attention. Twine and Ren’Py both seem to be solutions to the problem, except that they’re focused on their specific use-cases (generate an entire web-based game), and don’t seem easily reusable beyond that.

As I continued my research, I discovered Ink and Yarn Spinner, which are both designed as tools to embed in an existing project. Both are open-source projects. Ink was a better fit for me, but Yarn Spinner seems good too.

Basically, Ink does what I described before and the snippets above should be valid Ink code. But the language is much more powerful, with comprehensive documentation. The API is very simple: the game can request the next line of text from Ink as well as the list of choices.

Localization and Voice Overs

Although I’ve worked at Google, I sometimes care about localization, and my project requires translations and voice overs.

Some people complain that Ink doesn’t offer a native solution for that, but I’ve found it easy to implement translations on my side. Ink returns a list of strings to display. I just need to use the string as a lookup in my translation JSON file. So I built a JSON file that looks like this:

{
    "sentences": {
        "It was a good day.": {
            "key": "28-3",
            "en": "It was a good day.",
            "fr": "C'était une bonne journée.",
            "de": "Es war ein guter Tag.",
            "pl": "To był dobry dzień.",
            "sv": "Det var en bra dag.",
            "pt": "Foi um bom dia.",
            "nl": "Het was een goede dag."
        },
        ...

When Ink returns the string "It was a good day.", I just need to look it up in the JSON and get the translation. The field “key” is used for the voice over: the engine will play the file “28-3-fr.mp3” if the game is set in French.

Of course, the strings have to be static (Ink is able to compute strings and include variables), but that’s required for the voice over anyway. To extract the list of strings and build the JSON file, I’ve decided to wrap each string in backquotes in the Ink file. I can then extract them with a simple regex.

Not Just Text

Even if the narrative engine outputs text, this doesn’t mean that the game has to be text-only. Ultimately, the game engine can decide what to do with the string.

In my case, I use the string as a lookup in my translation table and find the corresponding audio, but there are many possibilities.

The string can contain meta-information, such as the name of the person speaking. But it could also contain machine instructions: the lines can be instructions for the game engine in order to trigger character animations, control the camera, or any other visual effect. We just need to write a parser for the output of Ink... Did I tell you I like creating languages?

On top of that, Ink can also modify variables. The game engine can “watch” them and run a callback when it happens, for example to display a health bar or represent any other value.

With all these features, we can make pretty complex choice-based, visual stories; but remember that many games offer players the freedom to explore…

Free Movement

If the narrative engine is just about lines of text and choices, how to handle free movement? Unlike linear narratives, many games don’t present a list of choices at each step. Instead, we need a way to trigger narrative nodes based on interactions. The solution can be quite simple.

If the game has a list of characters to talk to, we can create one section for each character. When the player talks to someone, we jump to their section and start the dialogue from there. Instead of having one giant graph with all actions in the game, we only have one small graph for each character. The graphs are not fully independent though: making a promise to a character might change a variable, which can then influence the discussion with someone else.

Some games have lots of items to interact with, with custom events. In that case, we can consider having a section for each item:

=== obj.chicken ===

* [Pick up]
      Maybe no one will miss just this one thing...
* [Look]
      A rubber chicken with a pulley in the middle… What possible use could that have?
* [Use with cable]
      ...

The choice can be selected implicitly by the game engine; we don’t have to present the choice to the user like in a text-based game. If the player tries to do an action that doesn’t map to an existing choice, we throw an exception say a generic line.

Conclusion

That concludes my exploration. I was pleasantly surprised to discover these narrative engines. They seem quite flexible, with good tooling, and available in multiple languages (C++, C#, JS…).

Now that I’ve found a good solution, I need to make progress on my project.

code