Project N - a.k.a. what Daggerfall totally *doesn’t* need? Ok, I’ll do exactly that!

[Arneb]

So, in the last few months I’ve worked on a DFU fork that should allow the game to read and menage an expanded map. The aim is (not original at all, I know) to get all the continent (and beyond) explorable. Given my very limited coding skills, the project hasn’t been exactly fast-proceeding up to this point, but I thought it would have been even harder (like, well beyond my capabilities). But thanks to the generous help of many nice people, I got out of many apparent dead ends.

The project isn’t obviously completed, far from it. But I think I’m on the right track.
So, what has been accomplished in these last weeks? Here we go:

1. MAPS.BSA, CLIMATE.PAK, POLITIC.PAK and WOODS.WLD have been converted to .json and then, through some trigonometric magic, had every pixel rotated by 30 degrees counter-clockwise around map pixel 0,0.
The result is an Iliac Bay as faithful to the original as integer-based math allows, but cardinally correct in its orientation.
Following suggestion from Burt, those .json were later converted to .png, and now DFU reads directly from those (well, actually it reads from some 128x128 tiles that were obtained from those, more on it later). This means, among other things, that now it's possible to create and modify region, climate and heightmap just by modifying those files and putting them back in the right folders (almost).



2. I reworked the travel map, since the original one was clearly useless at this point. The new travel map is a bit rough, but it's dynamically generated, allows to toggle region and climate areas and has no transition between overworld view, region view and zoomed view, it's a whole image that can be zoomed in and out as needed. Still needs fine-tuning, but it does what it's supposed to.



3. I coded an editor with some functionalities: there are a bunch of filters to toggle on/off the usual heightmap/climate/politic/location maps.



It's also possible to select a pixel with or without a location and then modify everything about it: add/remove a location, modify said location in almost every aspect, set which RMB or RDB blocks it is made of; regarding this last part, it's now possible to add to a location one or more custom blocks with a new block name, making it so that it won't substitute every other block with the same name in other locations.



There are also a region manager to alter a region's stats or to create a new one, and a faction manager to (guess what?) do the same, but with factions.



Last, there's a working functionality that allows to randomly generate all the locations in a given region given certain parameters.



4. I downloaded Transbot9's Tamriel Heightmap from Nexus (Skyrim mods) and, after some experiments, I reduced it to 7680x6144 pixels (yes, it was shrinked: the original is a whooping 20480x16385) and then brutally thrusted the Iliac Bay in its right place. I then proceeded to manually map climates and regions, trying to take inspiration from UESP Lore sections, even if regarding the politic map, the major reference was Arena's Tamriel map.



The result is certainly debatable but, as said before, everything can be changed quite easily.

5. Thanks to contributions by Ninelan, Carademono and SquidKamer, I was able to make the code recognize additional Climate Types and Villagers, both old and new. Now the system is open to new assets of this kind.



In conclusion,this is the project, in an early state of development: much has been done, but much more has still to be if it doesn't want to remain just an oversized version of Daggerfall. Therefore anyone willing to contribute will be warmly welcomed.

In the next post I'll list what my ideas are for the prosecution of the project, and what those willing can contribute to.

[Arneb]

I'd like for ProjectN to take a different path than DFU did: while DFU is an almost faithful port of Daggerfall, with every big change to the gameplay relying on mods, I'm not against "hardcoding" some major changes directly in the game.

For a series of reasons: first, I'm not expecting for this to get as much favour as DFU did and does, thus I don't expect to get that many "ProjectN-only" mods, if any at all. Second, from my point of view a world this huge encourages a certain type of gameplay, more survival-oriented and less "let's fast travel to the other part of the continent in a couple of clicks". Last, there are some DFU mods that I consider simply essential for a proper DFU experience, and I mean those mods that are so well integrated in the game that seem like they were made for the original game.

Ok, after this (honestly quite useless) introduction, I'd like to list a few things:

Stuff needed:

- First and foremost, graphical assets: building textures, nature billboards for new climate types, townsfolk and billboards of different races, etc. On top of that, townsfolk should be made in two versions at least, light and heavy dressed, since now i.e. a Redguard can be found in a High Rock village.

- Other types of various assets, like new RMBs, new building models, new RDBs.

- The location-generator names use a database of "name-parts" divided into regionalisms: at the moment we have those needed for High Rock, Hammerfell, Skyrim, Morrowind and Reachmen areas (these last three thanks to Ninelan). These can/should be expanded on, and we obviously need the other regionalisms from scratch.

- The same is true for Argonian character names, that right now use Imperial ones.

- The Imperial character race must be made.

On top of that, any gameplay modification is open for debate.

Edit:
Stuff I'm working on/I'll work on sooner or later:

- Fixing the calendar and time: I already made it so that every month doesn't start with the same day, but follows the normal weekday progression; I'd like to change dawn/sunset time and seasons based on climate and longitude/latitude. It shouldn't be too hard in theory, but I still have to experiment with that to see how the system takes it.

- Procedurally generated roads and rivers: Basic Roads is one of those mods I consider fundamental to DFU, but at the same time is one of those mods that is severely incompatible with this fork (for obvious reasons). On top of that, manually mapping every road in the continent is unrealistic, thus the best solution is to write a roads/rivers generator that takes into consideration altitude/climate/location connected etc.

[11th_defender]

So excited!

[jayhova]

This is the sort of thing I have wanted for a quarter century.

[Roranicus434]

Hey Arneb! "TheFafster" from Reddit here! Your work on this project so far is INCREDIBLE! Long have I dreamt of this day. I simply cannot believe how you managed to solve the problem of the rotation of the original map.

I had two things I wanted to bring up. In regards to texture and model assets, have you come across onasis101's Morrowind-Style Dwemer Ruin Assets posts here on the forums? I feel like they may be helpful for creating/generating Dwemer dungeons in the new expanded map.

Finally, in regards to location-generator names, I feel like I may be able to contribute. Though I am not entirely well-versed on how the location "name-parts" and regionalisms work (I'd love to know where I can find more info regarding this), I am a huge conlang fan, and I know there are several good resources on the UESP and Imperial Library that either document or expand on the individual TES race languages that may be used as reference or inspiration. If there are any guidelines you'd want me to follow, please let me know. Also, where should I share my contributions? Sorry for all the questions, but this is my first time interacting here on the forums and I am just very eager to help your project in any way I can!

[Arneb]

I had two things I wanted to bring up. In regards to texture and model assets, have you come across onasis101's Morrowind-Style Dwemer Ruin Assets posts here on the forums? I feel like they may be helpful for creating/generating Dwemer dungeons in the new expanded map.

Now that you mention it I remember reading that thread. Right now the problem is, while I already made some personal changes to how dungeons are generated in the fork, the basic system of dungeon blocks is still used. Therefore I see two ways of doing this: the first would be to make one or more totally custom dungeons with these assets and placing them somewhere, but it would be a drop in the ocean; the second, maybe more complex but better suited for Daggerfall (IMHO), way would be to create new dungeon blocks using those assets. Once those exists, I could insert them in the location generator, and make the algorithm pick those only when creating a "Dwemer Ruins" dungeon type (which would be a new dungeon type, perfectly possible to do, this being a fork).
Making new blocks isn't exactly easy stuff, and not just technically (the World Editor that comes with DFU is a pretty powerful tool for that kind of stuff, but requires some effort to be used proficiently). But who knows, maybe someday someone will find the right inspiration to make them.

Finally, in regards to location-generator names, I feel like I may be able to contribute. Though I am not entirely well-versed on how the location "name-parts" and regionalisms work (I'd love to know where I can find more info regarding this), I am a huge conlang fan, and I know there are several good resources on the UESP and Imperial Library that either document or expand on the individual TES race languages that may be used as reference or inspiration. If there are any guidelines you'd want me to follow, please let me know. Also, where should I share my contributions? Sorry for all the questions, but this is my first time interacting here on the forums and I am just very eager to help your project in any way I can!

From my POV, that is a perfect place to start contributing to the project. I can give you all the details needed, plus what has been already done, in dm on Discord if you have it. If not, just dm me here. And thanks for your generous offer.

[Daniel87]

- Fixing the calendar and time: I already made it so that every month doesn't start with the same day, but follows the normal weekday progression; I'd like to change dawn/sunset time and seasons based on climate and longitude/latitude. It shouldn't be too hard in theory, but I still have to experiment with that to see how the system takes it.

- Procedurally generated roads and rivers: Basic Roads is one of those mods I consider fundamental to DFU, but at the same time is one of those mods that is severely incompatible with this fork (for obvious reasons). On top of that, manually mapping every road in the continent is unrealistic, thus the best solution is to write a roads/rivers generator that takes into consideration altitude/climate/location connected etc.

I don't know if this helps, but look into the mod Wilderness Overhaul from Macadaynu and me. (I only did the initial algorithm to distribute plants and trees in a more natural pattern over the landscape, while he finished the 95% of the mod, applying this logict to all other climates and different altitudes).
In the scripts of our mod you should be able to see how we split up the different plants and trees into groups and families and distributed them depending on altitude. This might be very useful for you to create millions of square miles of natural looking landscapes in a single go.
You can actually use a more fractal approach with an iterative algorithm to spread these forests over the worldmap. If I remember it right, we only used basic perlin noise to do this, but with an iterative fractal noise you could create largscale patterns as well as small scale patterns in a single go.
It is important to consider climate and altitude aswell in the distribution. Also very important: Design Principles like "small, medium, large" where a certain distribution of small, medium and large features are clustered to create a natural look, ex. smaller pebbles surrounding a couple larger rocks that themselves are distributed around a large boulder. This is true for continent-sized distributions as it is for region and landscape sizes.

Creating river beds and lakes is more complicated. The worldmap has a uniform water level on ocean height, so creating mountain lakes is just not possible.
To create a river map, you would need to simulate erosion. There are a couple of tutorials out there on google that teach you easy approaches to approximate erosion and subtract it from the heightmap.
I think the climates mod did a great job in distributiong climate realistically over the landscape, maybe the code can be of help as well. You could also try and load your heightmaps into TerraformFX and calculate the erosion for waterways there. You will need the paid version for anythign beyond fullHD resolution though.

If that is done, the world would be ready to be populated.

Btw, amazing work and progress so far! This project looks very promising!

[Daniel87]

I had two things I wanted to bring up. In regards to texture and model assets, have you come across onasis101's Morrowind-Style Dwemer Ruin Assets posts here on the forums? I feel like they may be helpful for creating/generating Dwemer dungeons in the new expanded map.

Now that you mention it I remember reading that thread. Right now the problem is, while I already made some personal changes to how dungeons are generated in the fork, the basic system of dungeon blocks is still used. Therefore I see two ways of doing this: the first would be to make one or more totally custom dungeons with these assets and placing them somewhere, but it would be a drop in the ocean; the second, maybe more complex but better suited for Daggerfall (IMHO), way would be to create new dungeon blocks using those assets. Once those exists, I could insert them in the location generator, and make the algorithm pick those only when creating a "Dwemer Ruins" dungeon type (which would be a new dungeon type, perfectly possible to do, this being a fork).
Making new blocks isn't exactly easy stuff, and not just technically (the World Editor that comes with DFU is a pretty powerful tool for that kind of stuff, but requires some effort to be used proficiently). But who knows, maybe someday someone will find the right inspiration to make them.

Finally, in regards to location-generator names, I feel like I may be able to contribute. Though I am not entirely well-versed on how the location "name-parts" and regionalisms work (I'd love to know where I can find more info regarding this), I am a huge conlang fan, and I know there are several good resources on the UESP and Imperial Library that either document or expand on the individual TES race languages that may be used as reference or inspiration. If there are any guidelines you'd want me to follow, please let me know. Also, where should I share my contributions? Sorry for all the questions, but this is my first time interacting here on the forums and I am just very eager to help your project in any way I can!

From my POV, that is a perfect place to start contributing to the project. I can give you all the details needed, plus what has been already done, in dm on Discord if you have it. If not, just dm me here. And thanks for your generous offer.

The system used to create procedural dungeons wasn't too complex I believe. Just yesterday when clipping thrugh the walls, I could see all these loose dungeon ends that never got tied up. There are so many rooms floating in nothingness, because the algorithm creating the dungeons kind of was very flawed.
I remember seeing a Unity Tutorial series about a random dungeon generator that was very comprehensible.
https://www.youtube.com/results?search_ ... nerator+3d

Actually there are tons of tutorials on this, I bet you will create something that is even better than what Daggerfall used.

If you need somebody to quickly pop a 3D model into blender and change something, drop me a line and I will give it a try. (Hopefully I can import the model format into blender).
I worked quite extensively with blender, so quickly changing geometry is no problem at all (especially with these ridiculously low poly counts ^^)

[Jay_H]

Just yesterday when clipping thrugh the walls, I could see all these loose dungeon ends that never got tied up. There are so many rooms floating in nothingness, because the algorithm creating the dungeons kind of was very flawed.

Well, those aren't so much bugs as features. Those are called border blocks, and they're a necessary part of the generation. You can read more on why they exist here:

https://en.uesp.net/wiki/Daggerfall:Dun ... eon_Blocks

[Arneb]

I don't know if this helps, but look into the mod Wilderness Overhaul from Macadaynu and me[...]

WO is one of those mods I would like to hardcode in ProjectN. But I heard Macadaynu lost its source code in a harddisk failure. If that's actually true, we should probably need to remake it. In that case, your part of the project would probably be very useful.

Creating river beds and lakes is more complicated. The worldmap has a uniform water level on ocean height, so creating mountain lakes is just not possible.

Yeah, that's one of the limits of the original code, but this being a fork, I hope to be able to solve that problem (and many others: outdoor underwater sections among them).

Just yesterday when clipping thrugh the walls, I could see all these loose dungeon ends that never got tied up. There are so many rooms floating in nothingness, because the algorithm creating the dungeons kind of was very flawed.

Well, those aren't so much bugs as features. Those are called border blocks, and they're a necessary part of the generation. You can read more on why they exist here:

https://en.uesp.net/wiki/Daggerfall:Dun ... eon_Blocks

As Jehuty aptly pointed out, that's all part of how dungeons are generated. The only thing I changed involves how dungeon blocks are selected: my location generator, when building a dungeon, prefers blocks that, in an extremely subjective list I made, are listed as more appropriate for that dungeon type.
While right now the difference would not be too evident, when/if someone will decide to make new blocks, they can be "themed" and inserted in the correct block list.