Modding text macro handlers?

Discuss coding questions, pull requests, and implementation details.
Post Reply
User avatar
Kab the Bird Ranger
Posts: 12
Joined: Tue Feb 23, 2021 12:30 am

Modding text macro handlers?

Post by Kab the Bird Ranger »

I'm working on a mod to add alternative biographies and backstories for each class. For those interested in how that system works, I've documented all the backstories and all the strings involved on UESP here:
DFU does not support custom backstories right now, but I have a PR pending here: ... /pull/2065

These efforts have led me to wonder about adding new macro types in DFU (ex: "You swing your %foo."). Hazel has documented how the system works and how maintainers can improve on this here: viewtopic.php?f=23&t=673

However, as of now, it seems we can only add new macros by modifying DFU itself. While it's not really a blocker, it does mean new macros require sending a pull request and waiting for a new DFU release before mods can use them.

I'm wondering about what could be done to make the MCP system more mod-friendly. There's three main issues I see so far
1) handlers are set in a fixed, private dictionary inside MacroHelper. Mods cannot add new key-value pairs in there yet
2) for macros that require context, macro expansion relies on a "macro data source" (MDS) provided by a "macro context provider" (MCP) object. For example, the backstory strings will be expanded within the CreateCharBiography window with a BiogFile object, which has its own BiogFileMacroDataSource type. Mods cannot change the BiogFile object or its MDS, and therefore cannot store extra context that could be required by new macros, or change the behavior of macros
3) for each macro type, a new fixed function is added to the MacroDataSource interface. Adding new context-dependent macros requires modifying DFU code to modify that interface

Problem #1 is not too much of an issue in itself, just adding a public accessor would be enough.

Problem #2, in the current approach, probably requires work in every MCP class to make these mod-friendly. Not an insurmountable task, but worth noting.

Problem #3 probably cannot be easily worked around for modding without changing the approach.

For reference, here's a list of all the MCP classes: BiogFile, DaggerfallStats, DaggerfallUnityItem, IEntityEffect (implemented by BaseEntityEffect), Quest, and TalkManager.

Suggested solution

The first part of the solution solves both #1 and #3. Simply put, we add a method in MacroDataSource that takes the symbol string (ex: %foo) as a paramter and returns the expanded string (let's name it ExpandString as a placeholder). There's not really a need to have a fixed function for each macro type, only 2 macro handlers require any extra data to expand the string, but I think they can be ignored for this thread. In MacroHelper.GetValue(...), we will now get the MDS from the MCP and call ExpandString on it. If it returns null, we can call the existing macro handlers as a default. Note the order of the operations. This allows the MDS to handle macros that would otherwise be caught by the handlers - this is by design.

But just that is not good enough in itself, of course. For problem #2, we need a new type: IMacroDataSourceFactory. This interface has a factory function for each MCP (see list above). This factory will be set as a new field in DaggerfallUnity (along with similar objects, like textProvider), and mods can implement their own factory type and return their own macro data sources from there.

With these two changes, mods can override "ExpandString" to handle new macros, override existing macro functions to change its behavior, and add any extra data needed by the mod for context-awareness.

As an example, let's say my CustomBackgrounds mod wants to change the MDS used in BiogFile. During the Mod's Awake, it sets its own MacroDataSourceFactory in the DaggerfallUnity instance, which overrides ex: BiogFileDataSource() to return a new type derived from BiogFileMacroDataSource. When the time comes to expand a new character's backstory, BiogFile.GetMacroDataSource is called, which would now return DaggerfallUnity.Instance.macroDataSourceFactory.BiogFileDataSource(). The MacroHelper calls ExpandString on this MDS, and the mod can do whatever it wants there. For macros it does not want to override, it simply returns null, and everything keeps working as before, no extra work needed anywhere.

Design points
  • Why keep the old MDS functions at all, then?
We could certainly do a major refactor and put everything in ExpandString. This design was made so that minimal changes would be required. Each MCP only needs to be modified to use the factory, and maybe add a few protected accessors to the default MDSs so mods can reuse their code.
  • If the MDS factory is a singleton, doesn't that mean only a single mod can override it?
It's not any different from the textProvider. Mods can preserve the object set by previous mods or the default DFU one by using a simple "fallback chain" pattern. Here's what my mod does for the text provider

Code: Select all

public void Awake()
     ITextProvider currentTextProvider = DaggerfallUnity.Instance.TextProvider;
     DaggerfallUnity.Instance.TextProvider = new BackgroundTextProvider(currentTextProvider);
BackgroundTextProvider only overrides the functions it's interested in, and when it doesn't want to handle a certain case, it simply invokes whatever previous text provider was set, whether it's a modded one or the default one. The same pattern can be used for the MDS factory.

I hope the problem and the offered solution are clear enough from this post. In any case, I'll start working on an implementation, and will update this thread to link to it, so that readers may get a better idea of where I'm going.

Feel free to ask questions, or discuss any other issues.

User avatar
Kab the Bird Ranger
Posts: 12
Joined: Tue Feb 23, 2021 12:30 am

Re: Modding text macro handlers?

Post by Kab the Bird Ranger »

For context, I will link here, first, what the implementation of my solution basically looks like: ... at/mcp-mod

One thing I've noticed during implementing is that not all the relevant data in an MCP is public. But because the default MDS is implemented as an internal class, it has access to privates, so that's usually fine. When it comes to implementing a custom MDS for a mod, access to private fields from the MCP might be required.

Therefore, the implementation I linked above had to make decisions over what should be public in the MCP, and what should be exposed through a protected accessor in the MDS. I think I already covered every property that would be useful for mods.

Now, an example of a mod that uses this feature: ... cc5e02ff41

By using a custom TextProvider and the new MacroDataSourceFactory, I can make it so that my mod can replace or add new RSC strings associated with certain IDs, and as a result here, provide new backstories and new macros.

Here, I allow a backstory to have as many background characters as it wants. %b1n returns a deterministic random name based on the index 1, while %b17n would return a different name based on the index 17. I also provide gendered pronouns with a similar %b1g3, which matches the gender of the name returned by %b1n.

To clarify my intent with this post, as a mod such as this develops, more and more of such custom macros would be needed, and a mod should not have to submit pull requests to DFU and wait for a release every time it needs new text macros.

Post Reply