Next tasks?

Discuss coding questions, pull requests, and implementation details.
User avatar
InconsolableCellist
Posts: 67
Joined: Mon Mar 23, 2015 6:00 am

Re: Next tasks?

Post by InconsolableCellist » Wed Nov 09, 2016 8:25 pm

In addition to the previous ones, here are the updates that hook up the BookReader and fixes loading books from a save file for me:

https://github.com/InconsolableCellist/ ... 036e9bf6bc

I also added a CreateRandomBook() function, and hooked it into the LootTable in a way that should use the BK probability in a way similar to the other loot drops:

https://github.com/InconsolableCellist/ ... 8923bba05a

Here's the pull request that grabs all of these changes into your own book-updates branch:

https://github.com/Interkarma/daggerfall-unity/pull/152

I think a code review would be a good idea, and testing your own save file with a book in the inventory. The only thing left now is to create the mapping between books and filenames, which will allow the reader to read the actual book text instead of the test text.

User avatar
Interkarma
Posts: 2879
Joined: Sun Mar 22, 2015 1:51 am

Re: Next tasks?

Post by Interkarma » Wed Nov 09, 2016 11:16 pm

Awesome, thank you for your time and for the updates. :)
InconsolableCellist wrote: My investigation has led me to think the book and potion_recipe enums have the wrong offsets in their data structures, such that I think I have to change the template ordering so that an offset of 3 goes to book, 4 to potion recipe, and things like that.
You're correct both the index and order are important, and your changes shouldn't break anything. If your research shows that, I believe you. I'll definitely make the time to confirm as soon as I'm able. Could you please email/attach a zip of the save file you were testing books with? That makes it easier to compare against my own saves.

Cheers mate.

User avatar
InconsolableCellist
Posts: 67
Joined: Mon Mar 23, 2015 6:00 am

Re: Next tasks?

Post by InconsolableCellist » Wed Nov 09, 2016 11:34 pm

Sure thing, I'll PM you a link. For testing I recommend loading this save file and using "use" in the inventory screen on the books and parchment in the inventory. Expected behavior is a functional book reader for the book, and no action for the parchment. Same with a recipe. Quest notes I think have to show up in the inventory in another way? I had one in another save and it didn't show up as a book or parchment.

The text of the book will be The Real Barenziah still.

I also recommend spawning a new book somehow. I added CreateRandomBook() to the initial inventory code when you create a new char (not checked into git). I had to populate the thumbnail information for those to make them show up correctly.

It'd be helpful if you tested with your real saves too, both classic and DFU.

Up next I have to figure out what identifying information is in the save tree that lets me uniquely identify books. So far I was using the value of the book, which I can't imagine is guaranteed to be unique.

Thanks!

User avatar
Interkarma
Posts: 2879
Joined: Sun Mar 22, 2015 1:51 am

Re: Next tasks?

Post by Interkarma » Thu Nov 10, 2016 5:50 am

I found my research on books from earlier. Here's what I have.

Imported books have a "message" value in the item data. For example, "Confessions of a Thief" has message = 196664. The trick is to mask out lower 8 bits to get the book index. So bookIndex = message & 0xff... 196664 & 0xff = 56... Format this to BOK00056.TXT and you get the filename of the book, which is "Confessions of a Thief".

"Divad the Singer" has message = 196694. 196694 & 0xff = 86. BOK00086.TXT is "Divad the Singer". You get the idea. :)

I'm not sure what the upper 8 bits represents. Could just be "book" haha. I've always assumed that scrolls will index the spellid in the same way, but have not put this to the test at all.

When generating a new book, will need to set message value appropriately. Might need to enumerate BOK files and put some sanity checks in place to ensure random book goes to a valid file. And a BOK file dictionary would also give us a speedy way to match index value to title in tooltips.

Hope that helps!

User avatar
InconsolableCellist
Posts: 67
Joined: Mon Mar 23, 2015 6:00 am

Re: Next tasks?

Post by InconsolableCellist » Thu Nov 10, 2016 3:08 pm

Excellent! The "message" field was my last hope for a unique value, but I never would've thought it required masking. Is this a common thing in Daggerfall?

I had thought it'd be convenient to have a Dict<int, String> serve up the mapping between the two values, and as a data provider are you standardized on JSON data in txt files in the Resources folder?

Looking at LypyL's modding support post (viewtopic.php?f=4&t=277) it sounds like a mod can do any type of setup it wishes. So if someone wanted to override the book mapping and add their own information, they'd just have to create their own Dict and make sure the files are accessible. So I don't think my plan should interfere with potential mods.

Now I just have to figure out when and where it makes sense to populate this list based on the available BOK(\d+).TXT files


Thanks!

Edit: Actually, since you want a quick tooltip (a good idea), I'll do a Dict that gives more information, and create a function that can return a title for any ID without doing a lookup on disk

User avatar
Nystul
Posts: 1072
Joined: Mon Mar 23, 2015 8:31 am

Re: Next tasks?

Post by Nystul » Thu Nov 10, 2016 4:27 pm

bit offtopic, but since the mod system was mentioned:
@interkarma: I wondered lately if the mod system would allow to use new custom shaders flawlessly or if one would have the same problems we encountered recently (when I added new shaders to farterrain mod and reflections mod and those would not show up in the final builds until you did the workaround with the dummy-material).

User avatar
Interkarma
Posts: 2879
Joined: Sun Mar 22, 2015 1:51 am

Re: Next tasks?

Post by Interkarma » Thu Nov 10, 2016 10:12 pm

InconsolableCellist wrote:Excellent! The "message" field was my last hope for a unique value, but I never would've thought it required masking. Is this a common thing in Daggerfall?
Yeah, Julian loved his bitfields. :) If you see a weird 16 or 32-bit value somewhere, it's probably got at least 2 things packed into it.
InconsolableCellist wrote: I had thought it'd be convenient to have a Dict<int, String> serve up the mapping between the two values, and as a data provider are you standardized on JSON data in txt files in the Resources folder?
Pretty much standardized on JSON. Only real exception is the simplified data table format coming with quest system.
InconsolableCellist wrote: Looking at LypyL's modding support post (viewtopic.php?f=4&t=277) it sounds like a mod can do any type of setup it wishes. So if someone wanted to override the book mapping and add their own information, they'd just have to create their own Dict and make sure the files are accessible. So I don't think my plan should interfere with potential mods.
In theory, yep. When overriding default setup, just need some kind of hook for the mod (like an event) so it can do it's thing in place of the original. That's something mod creator can work out.
InconsolableCellist wrote: Now I just have to figure out when and where it makes sense to populate this list based on the available BOK(\d+).TXT files
You could possibly squeeze it into ItemHelper (as a new member, or just add the code to that class) which is part of DaggerfallUnity singleton and persistent for the entire session once instantiated (by accessing DaggerfallUnity.Instance.ItemHelper property first time). It should be quite fast to build an <index, info> dict on startup after enumerating files, maybe no need to serialize anything.

I don't have a problem if you feel it's best to do something else though. :)

User avatar
Interkarma
Posts: 2879
Joined: Sun Mar 22, 2015 1:51 am

Re: Next tasks?

Post by Interkarma » Thu Nov 10, 2016 10:24 pm

Nystul wrote:bit offtopic, but since the mod system was mentioned:
@interkarma: I wondered lately if the mod system would allow to use new custom shaders flawlessly or if one would have the same problems we encountered recently (when I added new shaders to farterrain mod and reflections mod and those would not show up in the final builds until you did the workaround with the dummy-material).
I will be honest, I can't answer this 100%. I think material shaders are built when the compiler imports the .dfmod assetpackage, so long as at least one material in the package has that shader attached. That's essentially what my dummy materials are doing, forcing Unity to build a specific variant of a material.

I'd love to speak with Lypyl to get some more training and insight on the mod system. It's an excellent foundation but no doubt will need work over time. I just hope he's OK and comes back someday. :cry:

User avatar
InconsolableCellist
Posts: 67
Joined: Mon Mar 23, 2015 6:00 am

Re: Next tasks?

Post by InconsolableCellist » Fri Nov 11, 2016 9:11 pm

I've found a bug: Daggerfall saves books with category2 values of both 2 AND 3. Previously this made some books show up as parchment (but with a book icon). I'll have to make a small change to differentiate between "real" parchment (the ones with the parchment icon) and this alternate type of book. In other words, two types of Book enums will be readable.

I'm adding the mapping stuff to make the real book text appear, as well as tooltips, and this fix will be in that commit.

User avatar
InconsolableCellist
Posts: 67
Joined: Mon Mar 23, 2015 6:00 am

Re: Next tasks?

Post by InconsolableCellist » Sat Nov 12, 2016 4:05 am

Okay! That bug is fixed and I also added ToolTips:

Code: Select all

Added tooltips to books, and a few book-related fixes

* Created a JSON file that contains the mapping between the book IDs and
the title of the book

* This file is now loaded into a Dictionary in the ItemHelper when that
class is first constructed. It then sits in memory

* A few helper functions get the names of books based on either the
decoded ID or the "message" field

* Fixes to the RandomBook function to make it work correctly. NOTE:
Added a TODO: FIXME: item regrading the unknown function of the upper
bits of the "message" field. Possible incompatibility when opening DFU
saves backin Daggerfall?
The last item in this list refers to "message" and the CreateRandomBook() function. In order to have a random book I have to pick a random message, but I don't know what the upper bits do. So I just pick a random ID and assign it to the message field. Since the code isn't using the higher order bits anywhere it won't break anything in DFU, but if you save the game and try to open it in DF it is technically not generating information that should be there. (Is this a thing that DFU intends to support?)

In another commit in the same book-updates branch I added a console command that came in handy for me:

Code: Select all

Added an 'add' console command to populate your inventory

* The new command is add (book|weapon|armor|cloth|ingr|relig) [n], where n
is the number of items to add
The changes are organized into four new commits. I made a pull request here:

https://github.com/Interkarma/daggerfall-unity/pull/153

I tested it a bunch, but of course a second pair of eyes on the code and testing would be good. Also take a look at the ItemHelper changes I made and make sure they're consistent with the proper way to do things in this codebase. I did try to follow the existing framework, however.

This should finish up the book feature, and the potion recipe and note readers should be pretty do-able after this.

Post Reply