Revisiting AI as a GM Support Tool

[Reynard]

The biggest limitations on using AI as a writing assistant is it is limited to just I think 14000 character buffers of what is in its "memory". So if you want to give it a large document as input, it will forget everything except the last 14000 characters. And if you have a long conversation with it, it won't take very long before it has lost all the context you carefully built up. It forgets what it is working on or what the rules were, and you continually have to remind it of the constraints you wanted to put on the content. I've used it to a limited extent, but it's not nearly as useful as I would like.

It's also not nearly as imaginative as it might first seem when speaking to it. And worse, it's output becomes part of its input so it rapidly overtrains itself and gets stuck in simple patterns.

This isn't true any longer, I don't think. I was listening to Hardfork and they were talking about ChatGPT4o having 2 million tokens for memory, and other LLMs providing similar numbers.

 

[Celebrim]

This isn't true any longer, I don't think. I was listening to Hardfork and they were talking about ChatGPT4o having 2 million tokens for memory, and other LLMs providing similar numbers.

That will definitely be a game changer. I haven't done a lot of testing on ChatGPT in a while, but I do remember concluding that it can't help you write a novel because it can't keep track of the overall plot or nature of the characters. It's actually not that good at short stories either. What I did find it good at was doing things like, "Rewrite this passage into a first person perspective."

 

[CapnZapp]

Let's be honest. A rpg adventure plot doesn't have to be fantastic, or deep, or unique, for the players to have a lot of fun for days on end.

Everybody will likely be using AI as a GM support tool very soon now.

 

[CapnZapp]

The biggest limitations on using AI as a writing assistant is it is limited to just I think 14000 character buffers of what is in its "memory". So if you want to give it a large document as input, it will forget everything except the last 14000 characters. And if you have a long conversation with it, it won't take very long before it has lost all the context you carefully built up. It forgets what it is working on or what the rules were, and you continually have to remind it of the constraints you wanted to put on the content. I've used it to a limited extent, but it's not nearly as useful as I would like.

It's also not nearly as imaginative as it might first seem when speaking to it. And worse, it's output becomes part of its input so it rapidly overtrains itself and gets stuck in simple patterns.

Get back to us next week...


AI development is a hyper frenzy right now. We're just starting. Every model you can use today will seem like a child's toy tomorrow.

 

[Celebrim]

Let's be honest. A rpg adventure plot doesn't have to be fantastic, or deep, or unique, for the players to have a lot of fun for days on end.

I think it depends on how long you've been playing.

I had this conversation at the last session I ran. As a GM, I know there are differences between my aesthetics of play and the players aesthetics of play. As a GM or player I lean toward more literary, more sand boxy, more investigative, more RP heavy play. I like tactical play and combat and plots and such, but that's not my core aesthetic. My players on the other hand like more linear, more straight foward, more combat heavy games were they kick down the doors, kill everyone, then find the clue that tells them which door to kick down next. So as I was giving them the set up to the second wide open sandbox adventure in a row, the last of which they got lost in for a long time before they figured out a solution, I was sort of apologetic about it and promised that the next time I'd run something straight forward and combat heavy but this time I wanted to do an adventure in space because it had been a while since they got the Z-95s out.

And one of my players said, "I don't mind just as long as the adventures continue to be varied".

So we're definitely reaching the point where he's played enough that they do need to be original and unique.

 

[Starfox]

I use AI in three ways.

1. As a suggestions box. "Give me ten random encounters in situation X"
2. As a secretary "Rewrite this text in the voice of X" or "Rewrite this at a lower reading level"
3. As an image generator. This is fun, but the results are bad. It takes hundreds of generated images to illustrate something specific.

 

[GrahamWills]

One thing to remember is that LLMs are really exceptionally new; there are a ton of questions we don't know about how to make them more useful or how different parameterizations affect the outcome.

What you see in ChatGPT is an extremely simple and generic solution for simple conversations. Which is quite different from what we might want our RPG AI to be.

Based on my work experience building LLM pipelines, if you wanted a solid tool for RPG use, I'd
suggest you would use 2 pipelines, one designed to create a vector-indexed database and use that with a fine-tuned LLM designed to answer questions about genre fiction; and a second one doing the same sort of thing for RPG rules.

You would then create databases of rules (using the second pipeline) for each system you establish a license with to do so (so relatively few of these), and allow users to upload their worlds into the first pipeline (and/or add licensed worlds).

For a query then, you would use the first pipeline to extract relevant fiction content from the world, and then pass those fiction chunks (using a meta-prompt) to the LLM with rules knowledge, which would use the fictional information and a meta-prompt that re-writes info that to fit the rules format.