The World Data Override system has been built so that mods can alter and add to the Daggerfall world data after it's streamed from the original data files, but before it's used by the DFU engine. This allows modifications of the gameworld without needing to change the original data files. The override files are json format and much easier to edit than the original binary files, and targetted sections of world data can be changed, while the remainder is unaltered. This also avoids the issue of these files being copyrighted by Bethseda, and the DFU model is to require the original files without modifications.
I know this tutorial is more of an info dump and is not easy to read & comprehend, but please do read it. I've tried to write down what I know in the best way I can and have spent hours doing this when I would much rather have been coding and solving problems. Making overrides will require effort, time and understanding. The more you're pushing the boundaries of what's already been done before, the more you will need to understand the data structures and the code that uses them - so you are aware of the various constraints and can problem solve to reach a solution. I'll help as much as i can, but I don't know everything and my answers will often involve me reading through code and figuring stuff out so you may need to be patient. (Hint: queries with all relevant detail and context provided, and demonstrating the effort taken to understand and solve yourself will always get a response )
What is World Data?
By world data I'm referring to the data that defines the locations of the Daggerfall game world. All the coloured pixels that you see on the games travel map and select as destinations to fast travel to. Basically towns, villages, dungeons etc. These locations are merged into the world terrain, which is a completely different system where a TerrainSampler uses the classic heightmap data to generate the terrain of the world. This terrain is not something that can be altered by the world data system discussed here. DFU basically flattens the terrain to place a locations' exterior and paints over the texture tiles of the terrain with texture tiles defined in the world data.
Much of the DFU code relies on the world data being as expected. There are constraints on what you can change without breaking DFU, so care needs to be taken to change the world data only in ways that DFU expects. Following existing patterns in the classic data is the safest way to ensure this.
Background info (must read and understand!)
The world of Daggerfall is specified in several bsa data files from the original game that DFU streams data from at runtime. These files define the overworld map, locations (towns & dungeons), buildings, dungeons etc that make up the gameworld. The world data override system allows selective parts of the location data to be overridden and changed.
There are over 15 thousand locations across the regions of the Iliac bay, constructed from reusable blocks. World blocks for the outside world (RMB - Record Map Block), which includes building interiors, and dungeon blocks for large interior spaces (RDB - Record Dungeon Block). Dungeons interiors are constructed from RDB blocks and towns & dungeon exteriors are constructed from RMB blocks. The locations are merged into the streaming world at the map pixel coordinates specified.
The hierarchy is illustrated in this diagram.
Each map pixel is mapped to a region, and each region has a bunch of locations within it. Each location then has an overworld exterior constructed from reusable RMB blocks. As you can see from the illustration blocks can be used in multiple locations, so changing a block will potentially change many locations across the world and in different regions since all places the block is used will change. Locations can also optionally define a dungeon from RDB blocks, again the blocks can be shared between different locations.
Console data dump commands
To assist with overriding world data I've added several data dump console commands to DFU that dump various world data into JSON files. The files go into the folder where you can find the DFU settings file and do nothing at all if left there. They are simply put there for you to look at, copy from, move and edit etc. Just be aware that the contents will include any override data the game has loaded from mods and loose assets, these will need to be disabled to get a file with just the vanilla data.
- dumpregion - Dump the current DFRegion to JSON file. (for example data only)
- dumplocation - Dump the current DFLocation to JSON file. (can be used for location overrides)
- dumpblock - Dump a set of RMB & RDB blocks to JSON files. e.g. "dumpblock RESIAM10.RMB M0000004.RDB" (can be used for RMB & RDB overrides) If no block names specified, will dump a block index to "BlockIndex.txt" containing all of the blocknames indexed as they are from the original data.
- dumplocblocks - Dump the names of blocks for each location, or locations for a block, to JSON file. With no argument you get a list of blocks for all locations in the game which will take a while so be patient. Providing a blockname gives a list of all locations split by region that contain that block. e.g. "dumplocblocks RESIAM10.RMB" (useful for finding everywhere a block you're modifying is used) Giving the argument "locindex" dumps a location index to "LocationIndex.txt" with regionIndex, locationIndex, location name.
- dumpbuilding - Dump the RmbSubRecord of current building player is inside to JSON file. e.g. "dumpbuilding"
Uncanny Valley has created a new locations modding resource already, and it's been used to create new things in the Daggerfall gameworld, so why is overriding world data of any use you may ask? The location loader and other resources in 'new locations' are fantastic, but they serve a different purpose. It enables new content made in Unity to be layered on top of the gameworld, allowing new decorations and places to appear in the exterior overworld. It doesn't enable new locations that can be fast travelled to or used in quest scripts however. I think the name locations is a bit misleading, since they're not daggerfall locations and are not integrated into the gameworld. Brilliant for adding small places of interest around the map, but not for a new town or dungeon.
Do I have to edit the raw json files?
Currently yes you do. Although Uncanny Valley has already made an building editor, and I believe has plans for a dungeon editor. Hopefully these will be released to the modding community and allow the json files for blocks to be editable using the Unity 3D editor. There will probably still be some json editing required, for locations for instance unless UV does an editor for that as well, but RMB and RDB edits where models and flats are being positioned will be a lot easier with a 3D editor.
What can I use to edit the raw json files?
Any text editor you like, but I would recommend one that can interpret json and allow section folding. This is incredibly helpful for navigating and focussing on specific parts of the data. My recommendation is to use VSCode which is free, provides json folding, and very good. Atom is another good option.
Development of override files.
When developing override files, the best way is to run the game in the Unity editor and put the files as loose files into StreamingAssets/WorldData/. This will allow changes to those files to be reflected in-game without requiring a restart. When not running in the editor, the override data will be loaded and cached for the rest of the session, so changes will not be seen in game until it's restarted.
I've attached a zip file with the test files I've used when developing this feature. The new dungeon block references a custom model from the Archaeologists mod, so you will need that installed or you will need to change the model id to one from the original data otherwise the block layout will fail. You can also see all the source files for my mods in github here: https://github.com/ajrb/dfunity-mods
Using Variants for a more dynamic Daggerfall world!
The world data variant system allows different overrides to be used once a trigger happens. The triggers can set variant blocks (or location / buildings) to be used just for a specific location, or every location using that block like the base system does. A variant is simply a string that selects a specific world data override file, for example in the Roleplay & Realism mod I use the variant "_master" to select a file called ARMRAM03.RMB-765-building14_master.json that changes building number 14 in the ARMRAM03.RMB block. I used the underscore prefix to make variant separate from the rest of the filename, but it's not required. The building is actually an armorer shop that was not given the right data to be a real shop, and the override file changes it into a very fancy & decorated shop with a special NPC.
The class that controls this is WorldDataVariants and has 4 methods for adding & changing variants as listed below. For blocks and buildings, a locationKey can be specified if you want the variant to only appear in a specific location. A location key is simply the location index combined with its region index and you should use WorldDataReplacement.MakeLocationKey(int regionIndex, int locationIndex) to create one. If you want a variant block or building to be used for all locations, don't supply a locationKey or use -8 (chosen because it looks like an infinity standing up, and I'm bored of using -1, do refer to the constant AnyLocationKey )
- SetLocationVariant(int regionIndex, int locationIndex, string variant)
- SetNewLocationVariant(int regionIndex, string locationName, string variant) - Need a different method for new locations as the index is only allocated at runtime.
- SetBlockVariant(string blockName, string variant, int locationKey = -8)
- SetBuildingVariant(string blockName, int recordIndex, string variant, int locationKey = -8)
There's a quest action for triggering variants called worldupdate and can take the following forms:
- worldupdate location at <locationIndex> in region <regionIndex> variant <variant>
- worldupdate locationnew named <locationName> in region <regionIndex> variant <variant>
- worldupdate block <blockName> at <locationIndex> in region <regionIndex> variant <variant>
- worldupdate blockAll <blockName> variant <variant>
- worldupdate building <blockName> <recordIndex> at <locationIndex> in region <regionIndex> variant <variant>
- worldupdate buildingAll <blockName> <recordIndex> variant <variant>
Code: Select all
worldupdate building ARMRAM03.RMB 14 at 240 in region 52 variant _master