Adding coastal locations using world data system (difficult)

[Hazelnut]

It it essential that these ports are fast travel destinations and available as quest locations? Maybe it would be useful to start a thread about what you're trying to do and context, to see what solutions may fit best given the constraints that exist.

There are a few issues, which I consider DFU engine bugs:

You can't break the system constraints and then claim when things don't work that they are engine bugs. They are not bugs because the input data is not within the range that the code expects. The world data system allows modders to selectively override the world data that DFU reads from the original files. That's it. This allows changing the world locations within the constraints of the engine, and that's all. It is not meant to allow everything to be possible, sorry if this is disappointing.

Obviously, as you suggest, you could develop an overhaul of DFU that will allow you to achieve what you want. You're most welcome to do that, but you'll be navigating new territory there and will not be able to rely on guidance from others.

EDIT: You need the archaeologists mod for the new model that is used in these test files. Did I forget to say that in my text? If so, sorry I thought I had mentioned it. Too busy with work to keep tabs on my DFU stuff right now.

[BadLuckBurt]

@Yagiza, I've had a look at the Port Daggerfall location and found the following issues:

NPCs can not be talked to, neither outside or inside buildings. I checked the log and it has to do with this error:

Spoiler!

GetRegionFaction() found more than 1 matching NPC faction for region 31.

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug.bindings.h Line: 43)

IndexOutOfRangeException: Array index is out of range.
at DaggerfallWorkshop.Game.Player.PersistentFactionData.GetRegionFaction (Int32 regionIndex, DaggerfallConnect.Arena2.FactionData& factionData, Boolean duplicateException) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.TalkManager.GetReactionToPlayer (SocialGroups socialGroup) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.TalkManager.TalkToMobileNPC (DaggerfallWorkshop.Game.MobilePersonNPC targetNPC) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.PlayerActivate.ActivateMobileNPC (RaycastHit hit, DaggerfallWorkshop.Game.MobilePersonNPC mobileNpc) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.PlayerActivate.Update () [0x00000] in <filename unknown>:0

(Filename: Line: -1)

Furthermore, if you fast travel to Daggerfall city and then fast travel back to the port, you will fall through the terrain. As long as you arrive south from the port it's fine but when the fast travel places you North, it's void time.

I agree with Hazelnut, what you encounter with the fast travel map not showing the pixels is not a bug, it's a limitation of how Daggerfall works. The High Rock Sea Coast region was never meant to be traveled to so forcing locations to be in that region breaks stuff.

Do a search in the codebase for 'RegionIndex' and see where it's used, you'll see how much would need to be changed to allow you to do what you want.

My advice would be to settle for the next best thing, place a location as close to the coast as it can go while still in a valid region and then use Kamer's technique for adding the port section.

[Yagiza]

There are a few issues, which I consider DFU engine bugs:

You can't break the system constraints and then claim when things don't work that they are engine bugs. They are not bugs because the input data is not within the range that the code expects. The world data system allows modders to selectively override the world data that DFU reads from the original files. That's it. This allows changing the world locations within the constraints of the engine, and that's all. It is not meant to allow everything to be possible, sorry if this is disappointing.

1. Incorrect display of player's position when it is in High Rock sea coast is obviously a bug, 'cause theer is no limitation for a player to be there.
2. Incorrect display of location name and no dot for locations in High Rock is a bug, cause Map displays those names and allows user to select those locations. If we decide to restrict user, we should not allow him to select locations outside of the current region and should not display names of such locations. If we allow and display incorrecltly, it's a bug.

Obviously, as you suggest, you could develop an overhaul of DFU that will allow you to achieve what you want. You're most welcome to do that, but you'll be navigating new territory there and will not be able to rely on guidance from others.

Yes. Right now I only see problems with world map pop-up windows. I suppose fixing those two classes will fix everything and should not cause any problem.

EDIT: You need the archaeologists mod for the new model that is used in these test files. Did I forget to say that in my text? If so, sorry I thought I had mentioned it. Too busy with work to keep tabs on my DFU stuff right now.

I didn't notice any mentions about it.
BTW, what's the difference between M0000004.RDB.json and original M0000004.RDB?

[Yagiza]

@Yagiza, I've had a look at the Port Daggerfall location and found the following issues:

NPCs can not be talked to, neither outside or inside buildings. I checked the log and it has to do with this error:

Spoiler!

GetRegionFaction() found more than 1 matching NPC faction for region 31.

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug.bindings.h Line: 43)

IndexOutOfRangeException: Array index is out of range.
at DaggerfallWorkshop.Game.Player.PersistentFactionData.GetRegionFaction (Int32 regionIndex, DaggerfallConnect.Arena2.FactionData& factionData, Boolean duplicateException) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.TalkManager.GetReactionToPlayer (SocialGroups socialGroup) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.TalkManager.TalkToMobileNPC (DaggerfallWorkshop.Game.MobilePersonNPC targetNPC) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.PlayerActivate.ActivateMobileNPC (RaycastHit hit, DaggerfallWorkshop.Game.MobilePersonNPC mobileNpc) [0x00000] in <filename unknown>:0
at DaggerfallWorkshop.Game.PlayerActivate.Update () [0x00000] in <filename unknown>:0

(Filename: Line: -1)

This issue is to be investigated. I guess, the problem is that there is no faction data for High Rock sea coast region. So, it could be easily fixed.

Furthermore, if you fast travel to Daggerfall city and then fast travel back to the port, you will fall through the terrain. As long as you arrive south from the port it's fine but when the fast travel places you North, it's void time.

Yes. That's what I mentioned in above. I think it could be easily fixed by correcting fast travel code in DaggerfallTravelPopUp class.

I agree with Hazelnut, what you encounter with the fast travel map not showing the pixels is not a bug, it's a limitation of how Daggerfall works. The High Rock Sea Coast region was never meant to be traveled to so forcing locations to be in that region breaks stuff.

Yes, we cannot fast travel to High Rock sea coast region? but we can easily go there by either walking or riding. So, incorrect display of player's position when he is in that region is obviously a bug. There was no such issue in original Daggerfall.
And again, when we talk about locations in regions, neighbour to the current one. It is not a bug that no dots displayed for those locations, when those locations are totally ignored. But if player can chose them, or if he can hover them and incorrect name is displayed at the top of the window - it seems like a bug for me. But I think instead of fixing such bug, it's much better to improve that behavior, to give mod developers more options.

Do a search in the codebase for 'RegionIndex' and see where it's used, you'll see how much would need to be changed to allow you to do what you want.

That's why I think that allowing to override pixel properties (like a region it belongs to or it's type) is much better solution. It should go seamlessly and should not break anyting in DFU engine.

My advice would be to settle for the next best thing, place a location as close to the coast as it can go while still in a valid region and then use Kamer's technique for adding the port section.

I'm sure this is not a solution at all.
First, the nearest pixel in Daggerfall region is at least 1 pixel away from the shoreline. So, instead of a small location port will become really huge. After fast travel to a port location, player will need to take a walk or a ride for almost 2 map pixels, before it arrive a ship in the sea! That's just awful.
Second, Kamer's LocationLoader do not cooperate with terrain sampler mods. So, once you enable, let's say, Distant Terrain, then go to the Sentinel harbor, you'll see that a ship sunk, so only top its sail is above the water!

[BadLuckBurt]

This issue is to be investigated. I guess, the problem is that there is no faction data for High Rock sea coast region. So, it could be easily fixed.

I have no idea, the error states that it finds more than 1 NPC faction so that's opposite of what you're saying. If you want to give fixing it a shot then do so by all means.

Furthermore, if you fast travel to Daggerfall city and then fast travel back to the port, you will fall through the terrain. As long as you arrive south from the port it's fine but when the fast travel places you North, it's void time.

Yes. That's what I mentioned in above. I think it could be easily fixed by correcting fast travel code in DaggerfallTravelPopUp class.

Probably true, I don't know a lot about terrains.

Yes, we cannot fast travel to High Rock sea coast region? but we can easily go there by either walking or riding. So, incorrect display of player's position when he is in that region is obviously a bug. There was no such issue in original Daggerfall.
And again, when we talk about locations in regions, neighbout to the current one. It is not a bug that no dots displayed for those locations, when those locations are totally ignored. But if player can chose them, or if he can hover them and incorrect name is displayed at the top of the window - it seems like a bug for me. But I think instead of fixing such bug, it's much better to improve that behavior, to give mod developers more options.

It's not a bug because you're going outside the original game data's boundaries with what you're doing. To handle your case, DFU needs to be extended. You can consider it a bug if you want to, I don't.

Do a search in the codebase for 'RegionIndex' and see where it's used, you'll see how much would need to be changed to allow you to do what you want.

That's why I think that allowing to overrid pixel properties (like a region it belongs to or it's type) is much better solution. It should go seamlessly and should not break anyting in DFU engine.

Have you done a search and seen what systems use the RegionIndex? Those would all need to be addressed,

My advice would be to settle for the next best thing, place a location as close to the coast as it can go while still in a valid region and then use Kamer's technique for adding the port section.

I'm sure this is not a solution at all.
First, the nearest pixel in Daggerfall region is 2 at least 1 pixel away from the shoreline. So, instead of a small location port will become really huge. After fast travel to a port location, player will need to take a walk or a ride for almost 2 map pixels, before it arrive a ship in the sea! That's just awful.
Second, Kamer's LocationLoader do not cooperate with terrain sampler mods. So, once you enable, let's say, Distant Terrain, then go to thw Sentinel harbor, you'll see that a ship sunk, so only top its sail is above the water!

The LocationLoader not conforming to terrain samplers is what I'd call a bug. I'm aware my suggestion has it's cons but there's enough to figure out about injecting locations without worrying about creating a new region and the data that needs to be associated as that's what you'll end up doing if you really want this in atm. I understand the appeal of what you want to do, I just think it's way too early when we still have duplicate interiors when we clone a location.

[Yagiza]

Do a search in the codebase for 'RegionIndex' and see where it's used, you'll see how much would need to be changed to allow you to do what you want.

That's why I think that allowing to overrid pixel properties (like a region it belongs to or it's type) is much better solution. It should go seamlessly and should not break anyting in DFU engine.

Have you done a search and seen what systems use the RegionIndex? Those would all need to be addressed,

No matter how many systems use RegionIndex. Why do you think that overriding it may affect them?
I didn't investigate the code deep enough, but I suppose that DFU has some king of wrapper around original Daggerfall data, which reads it and builds some structures, used internally by the game engine. So, if we override such data, after it was read, none of the game engine subsystem will notice it.
Or am I wrong and there is no wrapper? And all the game subsytems, which need original Daggerfall data read it directly? If so, it sounds somewhat irrational for me.

The LocationLoader not conforming to terrain samplers is what I'd call a bug. I'm aware my suggestion has it's cons but there's enough to figure out about injecting locations without worrying about creating a new region and the data that needs to be associated as that's what you'll end up doing if you really want this in atm. I understand the appeal of what you want to do, I just think it's way too early when we still have duplicate interiors when we clone a location.

Can you tell me more details about that interior duplication? How can I reproduce the bug?

[Yagiza]

BTW, what about dungeon entrances? I explored blocks and locations with and without dungeons, but I couldn’t find anything.

[BadLuckBurt]

No matter how many systems use RegionIndex. Why do you think that overriding it may affect them?
I didn't investigate the code deep enough, but I suppose that DFU has some king of wrapper around original Daggerfall data, which reads it and builds some structures, used internally by the game engine. So, if we override such data, after it was read, none of the game engine subsystem will notice it.
Or am I wrong and there is no wrapper? And all the game subsytems, which need original Daggerfall data read it directly? If so, it sounds somewhat irrational for me.

You'll have to take a look yourself, if you want to override that data I guess you can try but neither me or Hazelnut has touched on that so we can't help there. I just did a search for RegionIndex and saw that multiple systems reference it, including banks and knight guilds and decided to give you a headsup that if you're going to attempt this, to take that into account as well.

The LocationLoader not conforming to terrain samplers is what I'd call a bug. I'm aware my suggestion has it's cons but there's enough to figure out about injecting locations without worrying about creating a new region and the data that needs to be associated as that's what you'll end up doing if you really want this in atm. I understand the appeal of what you want to do, I just think it's way too early when we still have duplicate interiors when we clone a location.

Can you tell me more details about that interior duplication? How can I reproduce the bug?

If you make a new town exterior, all it's interiors will link back to the interiors of the original location. Just make a clone, go into the same building at the original location and the clone, use the 'dumpbuilding' command. You'll see the <blockname>.RMB-building-x.json that is dumped, it's the same for both locations. The NPCs in buildings will also have the same names.

[Hazelnut]

So I spent a few hours tonight looking at coastal locations and it seems there are a few issues with doing this, and is probably why classic doesn't really have locations right next to the sea.

The region boundary doesn't cover a lot of the coastal pixels for near Daggerfall at least. I think the only way to move forward with this is to be able to convert the map pixel for a new location to the daggerfall region, there are simply too many issues with trying to use a non-region like high rock coast for actual locations. (as you have found) Having spent some time looking at doing this, it's actually not as bad as I thought. In fact there are some non-joined pixels of an island off the coast, so I am pretty confident that just changing a specific pixel for a new location will be okay. The main issue would be caused by changing a pixel which already had a location on it, so don't do that. I will look at adding this but not right away, i want to get the existing PR merged and in builds before I add any more stuff.

For the time being, can you just add a pixel specific conditional in PlayerGPS after it gets the value from the politic pak file? I tried this tonight and it worked just fine, and will allow you to continue to make progress with developing your worlddata files until I add the ability to DFU code. Like this:

Code: Select all

currentPoliticIndex = dfUnity.ContentReader.MapFileReader.GetPoliticIndex(x, y);
if (x == 208 && y == 215)
	currentPoliticIndex = 145;	

The second issue is placement, and I couldn't improve on what you've done really. If the pixel is on land then the average height is too high, and unfortunately none of the coast exactly lines up with pixel borders. Currently I don't see how the port can be attached to the land without a small water channel in between. Do you have any thoughts on how you will handle this?

Third issue is the pixel is not coloured like it is on land. Have not figured this issue out yet, so please let me know if you figure it out.

Also, the tutorial is not a manual really. I've tried to write down what I've worked out, but there's a lot I don't know so please realise it's info as far as is known, and not a manual saying how to do what you may want to do. I have tried to give a rough idea of how to do various things in the hope it helps people out.

Lastly, I've split this discussion out into its own thread. I'd like to keep the tutorial thread just about clarifying the info in the tutorial. Thanks for understanding. (I hope)

[Yagiza]

Code: Select all

currentPoliticIndex = dfUnity.ContentReader.MapFileReader.GetPoliticIndex(x, y);
if (x == 208 && y == 215)
	currentPoliticIndex = 145;	

So, as I supposed, we have a wrapper class for reading Daggerfall from POLITIC.PAK, so pixel data could be easily overriden within that class, transparently to other parts of the game. That's nice.

The second issue is placement, and I couldn't improve on what you've done really. If the pixel is on land then the average height is too high, and unfortunately none of the coast exactly lines up with pixel borders. Currently I don't see how the port can be attached to the land without a small water channel in between. Do you have any thoughts on how you will handle this?

Third issue is the pixel is not coloured like it is on land. Have not figured this issue out yet, so please let me know if you figure it out.

The only idea I have for fixing both issues is improving terrain samplers the way, they take in account heighbour pixel blocks. The idea is that when sampler sees, that neighbour pixel is Water(Sea), but it has a location and its blocks, which, neighbour to this pixel are of Land type, stretch land part of the pixel to the border of it.

Before this is done, we can just do as this is done rigt now: use additional location, to force the Land pxel stretch to the neighbour Water pixel. And add some wet textures to the part of the block, which touches border of the pixel.