Re: Next tasks?
Posted: Sat Nov 12, 2016 5:33 am
Wow! Legendary work my friend. Looking forward to testing all this out!
Will do. I'll sit down to merge PR and review as soon as I get a bit of time. I will say I'm very impressed, and very grateful too.InconsolableCellist wrote:Great, thanks! Let me know if you have any questions.
Not yet, I had intended to be part of the spells and effects system which I'm planning to tackle post quests (probably by around mid 2017).InconsolableCellist wrote: For the potion stuff, is DFTFU already aware of the ingredients, their IDs, and how they map to the potion recipes?
The item templates were exported once only by ItemExplorerWindow.cs > ItemDataToJSON.cs > ItemsFile.cs. I had to post-fix several templates due to bad data on Daggerfall's side, so the JSON item templates now represent a cleaner copy of item data than Daggerfall itself.InconsolableCellist wrote: I see in FALL.EXE there are lists of ingredients and potion recipes. Are these imported somewhere? ItemTemplates.txt has ingredients in it, but I don't see a unique identifier from the game, which is contained somewhere in each potion record.
Does DFTFU do the importing from FALL.EXE, or is the information broken out using a third party utility and put into a JSON format or similar? I might have to reverse engineer the potion recipe information..
You found that as I was writing this reply haha.InconsolableCellist wrote:Re: reading from FALL.EXE, I found that it's doing a lot of what I need to be doing inside of the ReadFallExeFile() in ItemsFile.cs. It even says that it's starting at the Ruby object to determine the offset, which is what I was manually doing. So it's definitely reading the items, grabbing all available information, and putting it in the Items array.
Also your code has much more information than the UESP wiki...only one unknown value, whereas they have seven!
Thanks! Most of the hard work was done already thoughWill do. I'll sit down to merge PR and review as soon as I get a bit of time. I will say I'm very impressed, and very grateful too.
The quest system is more important, and I didn't know that the potion recipes were deferred (but it makes sense, given how it depends on ingredients)If you'd like to make a start on this, I'm happy to outline what I had in mind and you can run with it how you see fit. Otherwise you're welcome to come over and help with the quest system next. I'm sure we can find a way to mesh together despite the time zone differences. Any time you're willing to give to the project will be most welcome.
Code: Select all
* I manually exported the bytes in FALL.EXE that have strings that match the potion recipe names. Interestingly "Purification" (001B:A990) seems to prove that the string is last in the record, as no data follows that entry. I tried interpreting the records by both putting the string last in the record as well as first.
* My goal was to ferret out the bytes representing the ingredients. I figured it was represented in one byte without masking, and that they were sequential. (E.g., "this recipe requires pure water, nectar, and ectoplasm," with the ingredients being represented by three sequential bytes in any order)
* My method of attack was to write down all possible unique sequences of bytes in each record. So if a recipe called for four ingredients, I wrote down all sequential byte sequences that had four unique values. I did this for multiple recipes that shared ingredients, and my goal was to find the intersections. I.e., "suppose that <common ingredient 1> is represented by <value 1>. Is this value shared by all the other potions that have this ingredient? No? How about <value 2>?" And so on.
I wasn't able to map the ingredients in this way, which proves one of two things: the ingredients aren't contained in that section of data; the ingredients are encoded/masked.
My bet is that this data (001B:A130 - 001B:A9A0) is actually a table of potion/spell effects, and the effect names are referenced as table offsets elsewhere in the data. I'm not quite sure how to find this...maybe attach a debugger and look to see who accesses that table?
Was memory really that much of a premium, or were the developers just having fun with unnecessary optimizations? I really wish they'd release the Daggerfall source code. It might be that luciusDXL of XLEngine has figured some of this out, but my impression is that he wouldn't reveal it.Getting some of those unknowns was rough. Items have the most densely packed bitfields I've seen to date. Was very satisfying to nut them out though.
Excellent! Thank you for that. I haven't done much along these lines yet, very helpful. I'll keep this to refer back to.InconsolableCellist wrote: I'll briefly describe what I found from my investigation so far, as it may come in handy when this has to be tackled:
Code: Select all
* I manually exported the bytes in FALL.EXE that have strings that match the potion recipe names. Interestingly "Purification" (001B:A990) seems to prove that the string is last in the record, as no data follows that entry. I tried interpreting the records by both putting the string last in the record as well as first. * My goal was to ferret out the bytes representing the ingredients. I figured it was represented in one byte without masking, and that they were sequential. (E.g., "this recipe requires pure water, nectar, and ectoplasm," with the ingredients being represented by three sequential bytes in any order) * My method of attack was to write down all possible unique sequences of bytes in each record. So if a recipe called for four ingredients, I wrote down all sequential byte sequences that had four unique values. I did this for multiple recipes that shared ingredients, and my goal was to find the intersections. I.e., "suppose that <common ingredient 1> is represented by <value 1>. Is this value shared by all the other potions that have this ingredient? No? How about <value 2>?" And so on. I wasn't able to map the ingredients in this way, which proves one of two things: the ingredients aren't contained in that section of data; the ingredients are encoded/masked. My bet is that this data (001B:A130 - 001B:A9A0) is actually a table of potion/spell effects, and the effect names are referenced as table offsets elsewhere in the data. I'm not quite sure how to find this...maybe attach a debugger and look to see who accesses that table?
Haha, I often think that as well. I started development on a C64, so I know what it's like to scrape together every available byte - but man! I'm not sure the item templates had to be quite that packed. Or in the .exe for that matter.InconsolableCellist wrote: Was memory really that much of a premium, or were the developers just having fun with unnecessary optimizations? I really wish they'd release the Daggerfall source code. It might be that luciusDXL of XLEngine has figured some of this out, but my impression is that he wouldn't reveal it.
For sure! Right now, I've nearly finished the basics of parsing the .src structure into classes that will use that information (e.g. messages go to a Message class), and working on a quest debugger to show the components and state of active quests. I'm still a ways of doing serialization and a lot of other key things, but getting close to having some basics in place.InconsolableCellist wrote: Anyway! I'd be happy to contribute to the quest engine stuff, if you think there's a good chunk that could be broken off and worked on separately.
Sure! That sounds goodIf you can give me up to two weeks to wrap up what I have and post to git (I need to make some spare time), we can sit down together and knuckle out how we can divide and conquer.
Thanks! And your increased terrain distance improvements are awesome, and I'm glad to see them integrated into DFTFU/DFUI just want to chime in that I like how you collaborate here - it is nice to have another person involved. Can't wait to contribute as well (although I am completely unexperienced in reverse engineering unfortunately)
Code: Select all
width: 80
height: 256
compression: 0
pixDataLen: 20480
...
width: 256
height: 64
compression: 0
pixDataLen: 32002