Potions and ingredients

Discuss coding questions, pull requests, and implementation details.
Post Reply
User avatar
InconsolableCellist
Posts: 100
Joined: Mon Mar 23, 2015 6:00 am

Potions and ingredients

Post by InconsolableCellist »

I broke this topic off from the "next tasks?" thread to try and keep this one topic consolidated, but it builds off of information in that thread. I'll summarize the key findings that relate to the potions, ingredients, and recipe reader, and add more information:

The data from 001B:A130 - 001B:A9A0 in FALL.EXE contains a table of fixed-length structs that describe the potions in the game, but which don't seem to contain the actual ingredients used to make the potions. I've partially decoded the struct:

Code: Select all

// 109 bytes per record
struct potion { 

    char* ???[8];

    uint16_t value;

    char *???[24];

    char duration_min;
    char duration_max; 
    char duration_per_level;

    char *???[6];

    char chance_min;
    char chance_max;
    char chance_per_level;

    char *???[6];

    char magnitude_min;
    char magnitude_max;
    char magnitude_plus_min;
    char magnitude_plus_max; 
    char magnitude_per_level;

    char *???[10];

    const char* name[42];

}
I've done extensive fiddling to try and see if the unknown bytes contain the ingredients in any form, including bitwise operations to see if they're expressed as a bitfield (masked or not), or to see if they're indices into another table.

So far I'm led to conclude that the ingredients are described elsewhere, which probably means that another section of data has indices into the potions table. I also think that some of the unknown bytes in the potions table encodes spell effects, which will become important when the spell feature is tackled.

Ideally all of this information would be decoded from the EXE at runtime, but it may be necessary to just write the logic out into resource files and load them manually (basically plug in the information from the UESP wiki). That's what I'll end up doing to finish up the potion recipe reader, I think. Not as cool, but more practical.

I did one more thing to try and ferret out where the EXE might contain the table that indexes the above array of potion structs. I reasoned the following:
  • There'll be an array of unknown length, but with fixed-length structs
  • Each struct will contain an index into the potions table, which is 20 elements long
  • The structs will index the potions in order (similar to elsewhere in the EXE)
  • The struct size won't be bigger than 192 bytes, which is the item record size, I believe
I then wrote a program that recurses through the EXE to find an index, picks a record size, and looks for the next index. If it can find the next index, it repeats. If it can find all 20 indices in order without changing the record size, it prints this memory location out as an area that contains an array of fixed-length structures with indices that increment from 0 to 20 (or greater).

Here are those memory locations, though I don't expect them to be immediately useful to anyone. I'm including them because they could come in handy for implementing the spell feature: http://pastebin.com/TrgWFd6P. There are clearly some areas where font or printable character information is stored (which were flagged because it's highly sequential data)

More potion progress might not be possible until spells are taken care of. I think a good approach would be to visit a spellmaker and make a few spells with the same effects as a potion, then compare the saved bytes to reverse engineer the way the information is expressed. It could unlock the meaning of both spell records and potions. As seen on the roadmap though, this'll come after quests and some other features.

User avatar
InconsolableCellist
Posts: 100
Joined: Mon Mar 23, 2015 6:00 am

Re: Potions and ingredients

Post by InconsolableCellist »

Interkarma or Lypyl, could you shed some light on the way to get the IDs of the RSC text tokens? E.g., I see DaggerfallTravelMapWindow is using id 31 to get "Do you wish to travel to..." text, but I'm not sure how to do a lookup to find the text I need. The DFTEXT.EXE program to parse the RSC file also doesn't want to run on my modern system.

R.D.
Posts: 379
Joined: Fri Oct 07, 2016 10:41 am

Re: Potions and ingredients

Post by R.D. »

I'm not Interkarma or Lypyl, but I added that ID to DaggerfallTravelMapWindow. In that case, I just did so by comparing to the IDs used for the rest window popup, which are near to it.

You may already know this, but the messages in TEXT.RSC are divided by a particular token. If you look at it in a hex editor, it is FE in hexadecimal, or þ in ANSI. Counting the description for strength used in the character sheet window as 1, "Do you wish to travel to..." is after the 30th instance of the token. 1 + 30, so 31.

If you want to use a message that is near the beginning of the file, you could just manually count those tokens, but if it is far down, I guess you will need a better method.
Last edited by R.D. on Sun Dec 04, 2016 7:01 am, edited 1 time in total.

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Potions and ingredients

Post by Interkarma »

I've attached my spreadsheet of TEXT.RSC exported via DFTFU. The various formatting tokens are verbosely inlined (e.g. [0xFE]), rather than stored as bytes.

This is what I refer back to when hunting for a particular string. At runtime, Daggerfall Unity extracts directly from TEXT.RSC.
Attachments
AllText.zip
(118.66 KiB) Downloaded 199 times

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Potions and ingredients

Post by Interkarma »

Great work on the potion format so far. There might still be something interesting there. I'll jump in during the week after work and see if I can add to it at all. In any case, more information is always better than not. It's amazing how often a discovery made looking for something else will save time later. :)

User avatar
InconsolableCellist
Posts: 100
Joined: Mon Mar 23, 2015 6:00 am

Re: Potions and ingredients

Post by InconsolableCellist »

Thanks for the information R.D. and Interkarma. That spreadsheet is particularly useful. I knew DFTFU could parse the RSC file but wasn't sure if there was a convenient way to dump the data.

Curiously this particular string actually isn't in the RSC file, and is instead hardcoded into FALL.EXE. (001B:07B5):

Code: Select all

Recipe for %po(0x2E)(0x00)
Somewhere else in the code there must be a mapping of indices into this array of strings. (This sounds familiar.)

Is there an existing method for extracting these types of strings from the EXE, or should I just hardcode it into the potion recipe reader?

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Potions and ingredients

Post by Interkarma »

For now, I'm storing basic strings from the .exe in Game/UserInterfaceWindows/HardStrings.cs. I don't want to touch the .exe at runtime - there are too many different variants out there and we can't count on the offsets. Wider data (like the item templates and building name parts) are extracted to a standalone file, usually JSON.

At some point, all those strings will go into an external file. It would be awesome to unify all the text handling one day, but I'm not concerned at this point. :)

User avatar
InconsolableCellist
Posts: 100
Joined: Mon Mar 23, 2015 6:00 am

Re: Potions and ingredients

Post by InconsolableCellist »

Excellent! Thanks, the hardcoded strings worked great.

I finished the potion recipe reader:

Image

It looks like you don't have the "book-updates" branch anymore, so my pull request went to the master branch. If you create another branch again (just base it off master) I can issue another PR.

I described the changes in the commit messages. The latest one affects DaggerfallPopupWindow to let the caller override the alpha value for the dim color.

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Potions and ingredients

Post by Interkarma »

Perfect! Very exciting to have this working. I'm looking forward to checking out the potion data in more detail.

You rock, keep it up! :D

User avatar
InconsolableCellist
Posts: 100
Joined: Mon Mar 23, 2015 6:00 am

Re: Potions and ingredients

Post by InconsolableCellist »

Thanks! Let me know if you have any comments or need the PR for another branch.

The only piece left is the note reader, but I'm pretty sure notes are special inventory items that are placed there by quests. I had a quest-related note in my inventory but it doesn't show up in DFU. (Actually, maybe this is a bug in the load logic?)

We could also chat about quests and I could work on that when you're ready.

Post Reply