The World Data Override system has been built so that mods can alter and add to the Daggerfall world data after it's streamed from the original data files, but before it's used by the DFU engine. This allows modifications of the gameworld without needing to change the original data files. The override files are json format and much easier to edit than the original binary files, and targetted sections of world data can be changed, while the remainder is unaltered. This also avoids the issue of these files being copyrighted by Bethseda, and the DFU model is to require the original files without modifications.
Comparison with the 'New Locations' mod
Uncanny Valley has created a new locations modding resource already, and it's been used to create new things in the Daggerfall gameworld, so why is overriding world data of any use you may ask? The location loader and other resources in 'new locations' are fantastic, but they serve a different purpose. It enables new content made in Unity to be layered on top of the gameworld, allowing new decorations and places to appear in the exterior overworld. It doesn't enable new interiors that are transitioned into to be added. I think the name locations is a bit misleading, since they're not daggerfall locations and are not integrated into the gameworld. Brilliant for adding small places of interest around the map, but not for a new town or dungeon.
Background info (must read and understand!)
The world of Daggerfall is specified in several bsa data files from the original game that DFU streams data from at runtime. These files define the overworld map, locations (towns & dungeons), buildings, dungeons etc that make up the gameworld. The world data override system allows selective parts of the location data to be overridden and changed.
There are over 15 thousand locations across the regions of the Iliac bay, constructed from reusable blocks. World blocks for the outside world (RMB - Record Map Block), which includes building interiors, and dungeon blocks for large interior spaces (RDB - Record Dungeon Block). Dungeons interiors are constructed from RDB blocks and towns & dungeon exteriors are constructed from RMB blocks. The locations are merged into the streaming world at the map pixel coordinates specified.
The hierarchy is illustrated in this diagram.
Each map pixel is mapped to a region, and each region has a bunch of locations within it. Each location then has an overworld exterior constructed from reusable RMB blocks. As you can see from the illustration blocks can be used in multiple locations, so changing a block will potentially change many locations across the world and in different regions since all places the block is used will change. Locations can also optionally define a dungeon from RDB blocks, again the blocks can be shared between different locations.
Console data dump commands
To assist with overriding world data I've added several data dump console commands to DFU that dump various world data into JSON files. The files go into the folder where you can find the DFU settings file and do nothing at all if left there. They are simply put there for you to look at, copy from, move and edit etc. Just be aware that the contents will include any override data the game has loaded from mods and loose assets, these will need to be disabled to get a file with just the vanilla data.
- dumpregion - Dump the current DFRegion to JSON file. (for example data only)
- dumplocation - Dump the current DFLocation to JSON file. (can be used for location overrides)
- dumpblock - Dump a set of RMB & RDB blocks to JSON files. e.g. "dumpblock RESIAM10.RMB M0000004.RDB" (can be used for RDB overrides) If no block names specified, will dump a block index to "BlockIndex.txt" containing all of the blocknames indexed as they are from the original data.
- dumplocblocks - Dump the names of blocks for each location, or locations for a block, to JSON file. With no argument you get a list of blocks for all locations in the game which will take a while so be patient. Providing a blockname gives a list of all locations split by region that contain that block. e.g. "dumplocblocks RESIAM10.RMB" (useful for finding everywhere a block you're modifying is used) Giving the argument "locindex" dumps a location index to "LocationIndex.txt" with regionIndex, locationIndex, location name.
- dumpbuilding - Dump the RmbSubRecord of current building player is inside to JSON file. e.g. "dumpbuilding"
Currently yes you do. Although Uncanny Valley has already made an building editor, and I believe has plans for a dungeon editor. Hopefully these will be released to the modding community and allow the json files for blocks to be editable using the Unity 3D editor. There will probably still be some json editing required, for locations for instance unless UV does an editor for that as well, but RMB and RDB edits where models and flats are being positioned will be a lot easier with a 3D editor.
What can I use to edit the raw json files?
Any text editor you like, but I would recommend one that can interpret json and allow section folding. This is incredibly helpful for navigating and focussing on specific parts of the data. My recommendation is to use VSCode which is free, provides json folding, and very good. Atom is another good option.
Development of override files.
When developing override files, the best way is to run the game in the Unity editor and put the files as loose files into StreamingAssets/WorldData/. This will allow changes to those files to be reflected in-game without requiring a restart. When not running in the editor, the override data will be loaded and cached for the rest of the session, so changes will not be seen in game until it's restarted.
I've attached a zip file with the test files I've used when developing this feature. The new dungeon block references a custom model from the Archaeologists mod, so you will need that installed or you will need to change the model id to one from the original data otherwise the block layout will fail.
Using Variants for a more dynamic Daggerfall world! (WIP)
Coming soon is a world data variant system which will allow different overrides to be used once a trigger happens. The triggers can set variant blocks to be used just for a specific location, or every location using that block like the base system does.