[MOD] Unofficial Block, Location and Model Fixes

A curated forum for compatible and maintained mods. Users are unable to create new topics in this forum but can reply to existing topics. Please message a moderator to have your mod moved into this forum area.
Post Reply
User avatar
pango
Posts: 3347
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by pango »

XJDHDR wrote: Sun Jan 10, 2021 12:51 am I think it's an issue in the modding support. Those models look fine in the dungeon itself. I just tested this myself and there's something really strange about those models in the map. I saved my test save looking at the corridor model. After I loaded it, the corridor had the modded texture in the map whereas some other models I explored afterwards had no texture at all.
Mmmh reminds me of an issue / PR
However this time no exception is thrown (or at least not logged), so I'm not sure what's going on...
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

User avatar
XJDHDR
Posts: 258
Joined: Thu Jan 10, 2019 5:15 pm
Location: New Zealand
Contact:

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by XJDHDR »

jayhova wrote: Sat Jan 09, 2021 9:50 pm If it gets corrected in the DFU data it should be reasonably easy to reverse the process. In other words we have a system that decodes the original Daggerfall files it should be possible to edit and save that data back in the same format. If you could do that you could then compare the 2 files and create another patch file that would save the changes. It's all a bit over my head but it should be possible without having to hand-edit anything.

https://docs.moodle.org/dev/How_to_create_a_patch
As Pango said, DFU was only designed to read data from those BSAs. Someone would have to modify the code to write edited files back into them. The biggest issue I see is the parts of the BSA which no one has figured out what they do (yet?). I'm pretty sure DFU just ignores those unknown bits when writing out those json files, which means a writer would have to fill in the blanks. It gets trickier with the models though because Daggerfall Modelling converts the models it reads from the proprietary format DF uses into the Collada format. After editing it in Blender, I then saved them in the FBX and OBJ formats. Anyone wanting to put my fixes in the original BSA would have to convert those files back into Collada models (Blender is probably the best for that) then modify the DF Modelling code to write back into the BSA. It might be doable but someone would have to do the work required to make it happen and it's not going to be me.

The UESP has the best documentation I've seen so far on DF's BSA format. If you're interested, I included some links to the relevant pages in my readme under the FAQ section.
pango wrote: Sat Jan 09, 2021 10:02 pm In the existing bug reports, I've found only a few game-data related issues that were not fixed:
pango wrote: Sat Jan 09, 2021 10:09 pm More floating boxen (I'm not sure who contributed that save):
I've added these to the list as well.
pango wrote: Sat Jan 09, 2021 10:02 pm
My mod has already fixed this one. The block causing this is TVRNAM03.
pango wrote: Sat Jan 09, 2021 10:09 pm And that's most of the issues I was sitting on.
Along the way I noticed some of the bugs fixed by this mod, like stairs textures, really cool.
Thanks, that's good to hear.
pango wrote: Sun Jan 10, 2021 2:32 am Mmmh reminds me of an issue / PR
However this time no exception is thrown (or at least not logged), so I'm not sure what's going on...
I'll do some more tests to see if I can narrow it down.

User avatar
XJDHDR
Posts: 258
Joined: Thu Jan 10, 2019 5:15 pm
Location: New Zealand
Contact:

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by XJDHDR »

pango wrote: Sat Jan 09, 2021 7:21 pm There's some oddity though, and I don't know if that's an issue in the mod or in modding support: modified part of the cave looks different on the 3D map, it has a different texture and is not affected by transparency setting:
Did some more investigation into this. It looks like any form of model replacement causes those models to not have transparency.

As for the texture issues, it looks like a problem in Runtime Materials. In the image I attached, the opaque model in the middle was one I removed the Runtime Materials script from and applied materials to the regular way. After exploring it, it looks identical to it's appearance in the dungeon itself. The two white models on the left are ones I explored that still had the Runtime Materials script attached. The opaque model on the right also has the script attached but I was next to and looking at it when I loaded the game which meant it was already explored from the first time I opened the map.
Runtime Materials bug.jpg
Runtime Materials bug.jpg (98.72 KiB) Viewed 3573 times

User avatar
pango
Posts: 3347
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by pango »

If I follow the code correctly, I suspect it means modded sections have no "MeshRenderer" component?
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

User avatar
Ferital
Posts: 282
Joined: Thu Apr 05, 2018 8:01 am

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by Ferital »

Great work there :) Having included my own texture patch in DFU itself, I'm aware of the amount of work that this kind of patch require.
XJDHDR wrote: Sun Jan 10, 2021 8:46 pm As Pango said, DFU was only designed to read data from those BSAs. Someone would have to modify the code to write edited files back into them. The biggest issue I see is the parts of the BSA which no one has figured out what they do (yet?). I'm pretty sure DFU just ignores those unknown bits when writing out those json files, which means a writer would have to fill in the blanks. It gets trickier with the models though because Daggerfall Modelling converts the models it reads from the proprietary format DF uses into the Collada format. After editing it in Blender, I then saved them in the FBX and OBJ formats. Anyone wanting to put my fixes in the original BSA would have to convert those files back into Collada models (Blender is probably the best for that) then modify the DF Modelling code to write back into the BSA. It might be doable but someone would have to do the work required to make it happen and it's not going to be me.
There are part of this patch that could be applied to classic, provided some routines to convert extracted and patched data back to BSA binary format. This is exactly what I do for my texture patch: I wrote some functions to handle this in order for DF Modelling to generate the byte sequence to apply at a given offset. I could extend this to more stuff than texturing.

However, there are limitations to things we can do with BSA files. We can patch bytes, but not insert or delete them. Technically, we could do it but this would be very difficult since it would require to modify many places in the file since a BSA is nothing more than a packed collection of records. So this would require to extract a BSA file to a textual format such as JSON, including all fields, even unknown ones, patch this JSON export and convert it back to BSA format. And this would indeed require a lot of time.

User avatar
XJDHDR
Posts: 258
Joined: Thu Jan 10, 2019 5:15 pm
Location: New Zealand
Contact:

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by XJDHDR »

pango wrote: Mon Jan 11, 2021 10:41 am If I follow the code correctly, I suspect it means modded sections have no "MeshRenderer" component?
All of my models do already have a "MeshRenderer" component. I did discover something that might help find the problem though. All of the models in my screenshot were exported as OBJs. I tried exporting the staircase as an FBX then used that to replace the original OBJ model. This allowed transparency to work on this model. See this screenshot where the 3 stairs, using the FBX model, have working transparency whereas the corridor model, which is still using an OBJ model, is still opaque.
Transparency FBX test.jpg
Transparency FBX test.jpg (116.61 KiB) Viewed 3527 times

I suspect that what is going on here is similar to what TheLacus discovered while creating his Runtime Material script: His script initially only worked when attached to the root of the prefab whereas OBJs arranged things such that it needed to be attached to a child of the root. Maybe there is a similar problem going on here? The scripts are only looking for a "MeshRenderer" component on the root of the prefab? Here's a screenshot of how the two staircase prefabs were arranged. On the left is the FBX prefab and the right has the OBJ prefab. See how the OBJ has everything in the "default" child.
FBX vs OBJ prefabs.jpg
FBX vs OBJ prefabs.jpg (315.86 KiB) Viewed 3527 times

User avatar
pango
Posts: 3347
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by pango »

I assume the code could be changed to update the shaders of all MeshRenderers, recursively...

Edit: seems to work fine, I submitted a PR
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

User avatar
XJDHDR
Posts: 258
Joined: Thu Jan 10, 2019 5:15 pm
Location: New Zealand
Contact:

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by XJDHDR »

Ferital wrote: Mon Jan 11, 2021 3:16 pm Great work there :) Having included my own texture patch in DFU itself, I'm aware of the amount of work that this kind of patch require.

There are part of this patch that could be applied to classic, provided some routines to convert extracted and patched data back to BSA binary format. This is exactly what I do for my texture patch: I wrote some functions to handle this in order for DF Modelling to generate the byte sequence to apply at a given offset. I could extend this to more stuff than texturing.

However, there are limitations to things we can do with BSA files. We can patch bytes, but not insert or delete them. Technically, we could do it but this would be very difficult since it would require to modify many places in the file since a BSA is nothing more than a packed collection of records. So this would require to extract a BSA file to a textual format such as JSON, including all fields, even unknown ones, patch this JSON export and convert it back to BSA format. And this would indeed require a lot of time.
Thanks and pretty much agree with you there.
pango wrote: Mon Jan 11, 2021 10:37 pm I assume the code could be changed to update the shaders of all MeshRenderers, recursively...

Edit: seems to work fine, I submitted a PR
Thanks for fixing this. It's also working on my end. Hopefully, this can make it into the next release.

User avatar
Baler
Posts: 225
Joined: Thu May 23, 2019 1:39 am
Location: Earth

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by Baler »

This is so cool, the gaps between stairs, for example, was such a huge turn off for me. It felt like one of those rough edges of dfu, so glad to see it being resolved.

13thsyndicate
Posts: 75
Joined: Sat Jul 25, 2020 5:53 pm

Re: [MOD] Unofficial Block, Location and Model Fixes

Post by 13thsyndicate »

I don't know if you've fixed this one yet, but here's a bookstore in Midbury that's labeled as a residence.
residential bookstore.png
residential bookstore.png (1.52 MiB) Viewed 3445 times
Meanwhile, the actual bookstore behind it, marked orange on the map, doesn't have a bookstore sign, so I think somehow some data got switched?
Attachments
bookstore with no sign.jpg
bookstore with no sign.jpg (652.63 KiB) Viewed 3440 times

Post Reply