True Radiant quests with GPT and LLMs?

[thiseggowaffles]

Back to the use of language learning models in Daggerfall. You seem to be very excited about the very possibility of chatting with NPCs. I've been playing this game from time to time since the early 2010s I guess, but personally I've never felt the need for such talk -- although admittedly the built-in "dialogue" system in Daggerfall is incredibly limited even by the standards of early-90s (or even late-80s) CRPGs. I would say that this is a forgivable flaw because the game is slanted towards more action and the player generally being on their own, rather than character interaction. I would probably not mind a kind of dialogue system that would allow the player to discover more about the game world and lore, for example by typing keywords as in more traditional RPGs. At present I cannot really imagine any gameplay benefit besides being simply a gimmick feature for AI-driven conversations in Daggerfall. You can always prove me wrong if you provide some examples of how you view this.

Daggerfall's dialogue system is its greatest weakness. What you refer to as a gimmick would be a significant draw for many players who want a fully immersive RPG simulation experience. Daggerfall already does a great job of simulating the game world in a believable way. GPT API integration for conversing with NPCs, especially if you were able to pass contextual information from the game, would be an incredibly immersive experience.

[thiseggowaffles]

Let me feed your imagination. As these systems miniaturize, they can run natively inside the application or as a background process instead of having to interface with an API and query OpenAI with a delayed prompt-and-response across the Internet. That means less latency, which means no "Let me think," which means free-form and fully interactive conversations with an entity that has human-level intelligence. Something like Meta's leaked LLaMA could probably be adapted to a task like this already. If you train the model on a diet of Elder Scrolls fantasy and lore, give it some RLHF, and create overriding archetypes and rulesets for law-abiding NPCs, you can pretty easily get them to not talk about things like skooma. These are really miniscule, solvable problems. The point isn't that it's ready for primetime now, but this *is* the future of video games, and probably the future of all human-computer interaction eventually.

You are correct. I had the exact same thoughts. Someone could definitely train a version of LLaMA specifically on Elder Scrolls lore to be able to do this natively. It would require a good video card for users, but it could be done. I wish I had the relevant skills to do this tbh, but it's well outside of my current capabilities. This is 100% the future of gaming. I think Daggerfall would be an ideal testbed game, as it's already an incredibly detailed fantasy world simulation like Dwarf Fortress but with more direct player agency. If you could find a way to pass along contextual information to an LLM and vector database integration with something like Pinecone, you might be able to create NPCs that you could interact with in believable ways and establish relationships with them. The advantage to doing GPT integration instead of LLaMA integration though would be that GPT-3 and GPT-4 already have a certain degree of understanding of Daggerfall and the lore.

[MrFlibble]

Daggerfall's dialogue system is its greatest weakness. What you refer to as a gimmick would be a significant draw for many players who want a fully immersive RPG simulation experience.

I'd actually say not only the dialogue system (there are pre-release promo screenshots were you can see something more like the Morrowind dialogue system, which was never implemented), but most of the the NPCs themselves are problematic.

Enter any guild or palace, you see crowds of people with unique names, but save for the guild services and quests that they might give, they all have the same set of dialogue options, namely the generic stuff everyone has. Contrast this with I guess any other RPG of any kind, where NPCs usually serve different purposes like feeding the player background/lore information, advancing the plot or contributing to the immersion and atmosphere in some meaningful way. There's a reason why you couldn't talk to palace guards in Arena, because it'd have been too costly to make them into proper characters, and it probably could break suspension of disbelief if they just had generic dialogue.

I'd be quite interested to see the AI make each and every Daggerfall NPC unique in a way that would contribute to gameplay enhancement and immersion. Just note that, while the AI can more or less simulate a real person, in my opinion, NPCs are never meant to be real people. They're supposed to create an illusion of being real, but only inasmuch as it serves the overarching purpose of advancing the narrative. I suppose that even in tabletop RPGs, a DM would not be happy with players going for idle chit-chat with random NPCs, unless this is done in a measured way for the sake of acting in character.

I also do wonder how successful this hypothetical AI would be in making every NPC unique, and what gameplay enhancements could this entail. For example, when I imagine my character as a realistically acting person coming to a palace court in let's say Wayrest for the first time, I suppose that before addressing Princess Morgiah directly, I'd probably look for a courtier looking friendly enough and try to carefully ask about the other people in the throne room, gathering some light intel on these people, possibly some court gossip and work opportunities. In fact, even before that, I'd try local innkeeps, passers-by and perhaps local fellow members of guilds and temples that I might be part of to know more about the region, its rulers and relations with Daggerfall, before venturing into Castle Wayrest.

If the hypothetical AI is capable of maintaining a consistent and lore-accurate narrative for the player in such a situation at least within one playing session, I suppose that would be most laudable and desirable, and contribute directly to the roleplaying experience. But if it's only just some irrelevant small talk that it can produce, then I'd say it's a nice gimmick but not really doing anything for the game on the whole.

[thiseggowaffles]

If the hypothetical AI is capable of maintaining a consistent and lore-accurate narrative for the player in such a situation at least within one playing session, I suppose that would be most laudable and desirable, and contribute directly to the roleplaying experience.

Yeah, that would be the desired end-goal. You could train a LLM like LLaMA specifically with Elder Scrolls lore (ideally Daggerfall specific lore) and information from the wiki. Pass along contextual information from the game engine (such as name of the NPC, region, weather, etc) and then have the LLM chat specifically as that NPC. You would want to use a vector database like Pinecone for messages that are determined to be valuable for long-term memory, so that you the NPCs could remember you from session to session. Ideally I would want to integrate directly into game mechanics such that the NPC would attack if angered, etc. In theory, it's all possible and would greatly increase immersion, but it's far beyond my capabilities. That said, similar things have been done in other games (Skyrim, Rimworld, etc) using the ChatGPT API, so someone else more capable could likely do it. I tried to get an AutoGPT to do it for me, but the problem is that it can't analyze the source code due to token size limitations.

[King of Worms]

[Hurricane Otter]

Here's a great video from Nvidia. This is the kind of tech that's coming down the pike. The voice synth is a little lifeless, but the important bit is how seamlessly the LLM integrates into the gameplay loop. Implementing this wouldn't be trivial, but it would be far simpler in a game like Daggerfall where you don't have to worry about speech-to-text or procedural voice generation. All you need is the right LLM model and someone who knows the first thing about Unity modding... which isn't me, unfortunately, or else I'd have something in alpha already.

https://www.reddit.com/r/singularity/co ... _industry/

[Hurricane Otter]

I'm also thinking about the possible ethical concerns related to using AIs. In a post above you stated that you envision future AIs as effectively gaining human-level sentience if I understood this correctly. Would this not mean that you'd need to make sure that such an AI willingly participates in roles like being a perpetual DM in a video game, and how exactly is this going to be determined? Or the question of supposed free will and consent of AIs not a real issue?

That's a whooooole 'nother can of worms right there. I would argue in order for a thing to be sentient, it needs to be intelligent, self-aware, and conscious. These chatbots are obviously extremely intelligent. The jury's still out on the latter couple things — though there are prominent voices in the AI community that are open to the possibility.

I've used GPT-4 a lot. Based on my own experience, I would judge it to be an extremely intelligent zombie. I acknowledge that I might be wrong.

How can I know for sure? How can anyone know, really? How can I even be sure the human beings I interact with everyday are sentient? Maybe I'm just a brain in a tank somewhere, and everyone I know and love is just a simulation cooked up by some insane computer. If such solipsism means I can't even be 100% certain about my own friends and relatives, I sure as hell can't be sure about an AI.

Trust me, I've thought a LOT about this. All those philosophy classes I was sure were bullshit back in college are suddenly relevant.

I think, in the next fifteen years, probably sooner, someone will create a superintelligent AI that meets the criteria for sentience. I don't think we're there yet, but I expect the line to get pretty blurry. If you follow the current trendline, it's likely more and more emergent capabilities will manifest as the tech improves. This is stacked exponential acceleration: exponential advances in neural nets on top of exponential advances in hardware (Moore's Law). Who knows how long it'll take? In the meantime, I think we can make Daggerfall pretty great without too much of an ethical dilemma.

[MrFlibble]

I guess you'll know for sure it's sentient when it tells you to f-off because it does not want to play your games with you.

[Hurricane Otter]

Another example of LLMs moving into the indie scene:

https://twitter.com/DeveloperHarris/sta ... 7090353152