0.7.2: Quest timings

Discuss coding questions, pull requests, and implementation details.
User avatar
Interkarma
Posts: 4005
Joined: Sun Mar 22, 2015 1:51 am

Re: 0.7.2: Quest timings

Post by Interkarma » Thu Jan 03, 2019 11:26 pm

I'm definitely interested in exploring a new resource with the express purpose of better emulation of timers in classic quests. I also appreciate all the time you've spent picking all this apart.

Would you be interested in creating a bulk script to export those timers for all the classic quest files matching your RFC? Then it becomes an issue of updating those lines in the quest scripts and testing execution of each quest again. Oh, and writing the new timer resource itself of course. :)

Just a couple of minor items. I'd prefer the new resource be called something like "ClassicClock" rather than "Clock2", as this also describes the purpose. And the symbol export should keep the underscores (e.g. _delay_) just to remain consistent with rest of file exported by TEMPLATE.

I also have a quick question about the link values:

Code: Select all

Clock2 questime TIMER_2REF_2DEST(5?) 0 0 flags 0301(769?) links 2 1 
What are these actually referencing? Is it just an index into the list of Person resources (if an NPC) or Place resources (if a location). If so, is the index zero-based?

I can't say when I'll loop back to work on the new timer resource itself, but if we have the new timer definitions exported then at least this can progress over time. I'd also be happy for yourself and/or Pango to work on this further and send a PR - if that's an interest either of you have of course. My only request is there's minimum touch to core outside of implementing new timer class and updating Clock definitions in classic quest scripts.

Cheers for all the information! :)

Elenwel
Posts: 5
Joined: Tue Jan 01, 2019 5:14 pm

Re: 0.7.2: Quest timings

Post by Elenwel » Fri Jan 04, 2019 6:45 pm

Hi !

You can find my current script on https://github.com/TogDu/DaggerfallTools, and a sample output file here :
https://github.com/TogDu/DaggerfallTool ... AsTest.txt

For readability I chose to replace link:[id] [id] by ref:NAME or refs:NAME NAME. It may be easier to use for quest writers, but harder to parse. I can easily revert to the numerical ID version. By the way yes, this ids are indexes into Place/Person arrays, which are zero based.

You will see that some timers are ill named (UNKNOW_TVH tagged). In those cases Template use _S.XX_ placeholders, which seems to be sequentially allocated on a given quest.txt file. But those tags are not shared between files... For example $CUREVAM.QBN/_UNKNAME_00006CC3_ seems to be _S.13_ in $CUREVAM.txt but _S.05_ in $CUREWER.txt...

As those name are used to associate a Timer with its corresponding task, I will have to find a way to duplicate Templates name.

User avatar
pango
Posts: 789
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: 0.7.2: Quest timings

Post by pango » Sat Jan 05, 2019 12:31 am

Hi Elenwel,
Thank very much for all this info!
Personally I'd go for a more english-like syntax, closer to what is used for the rest of decompiled quests:

Code: Select all

type 0: 
ClassicClock _name_ <min> to <max> flags <flags>
type 1: 
ClassicClock _name_ <duration> flags <flags>
type 2 and 4: 
ClassicClock _name_ [(<duration>|<min> to <max>) plus] time to _name1_ [and back] flags <flags&0xef>
type 3: 
ClassicClock _name_ [(<duration>|<min> to <max>) plus] time from _name1_ to _name2_ [and back] flags <flags&0xef>
type 5: 
ClassicClock _name_ [(<duration>|<min> to <max>) plus] time to _name1_ then to _name2_ [and back] flags <flags&0xef>
durations being written as [[<day>.]<hours>:]<minutes> like today; That's hopefully parseable without too much trouble. Some stopwords can be made optional. I considered adding "npc" in front of names to distinguish locations and npcs, but no quest uses the same identifier for both, that should be redundant in text format.
So all cases can be parsed as:

Code: Select all

ClassicClock _name_ [<min> [to <max>]] [plus] [time (from|to) _name1_ [[then] to _name2_] [and back]] flags <flags>
and maybe even:

Code: Select all

ClassicClock _name_ [<min> [to <max>]] [plus] [time (from|to) _name1_ [[then] to _nameX_]* [and back]] flags <flags>
I tried merging this with existing decompiled quests using a script to keep the clock names, under the assumption that their order was preserved, which seems to be the case.
Got some surprises though, some quests changed names, and some have one more clock than what your script found (A0C00Y06, K0C00Y05, R0C11Y26, K0C0Y03,...). A0C00Y00 ClassicClock references _off_ that's not defined in the quest, same for B0C00Y05 about _qgiven_...
Maybe we're not all using quests patched to the same level or something (hash collisions?).
So I'm double checking everything by hand, it'll take some time :o
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

Elenwel
Posts: 5
Joined: Tue Jan 01, 2019 5:14 pm

Re: 0.7.2: Quest timings

Post by Elenwel » Sat Jan 05, 2019 9:00 am

I'm not a big fan of "natural language" representation. Even if I'm pretty confident with my current assumptions, those remain assumptions. For example I'm pretty sure that time min an max are just dropped out for destination based timer, but I chose to keep it in my current format. To avoid another global quest rewrite if someone else come with a better understanding.

Side note : QBN file format is far from being size efficient, there is a lot of unused, junk and odd fields. We have to keep in mind that it's a 1996 codebase :) . And it's stuffed with vulnerabilities too...

Ideally, I would have chosen a JSON or XML like representation for quest file, it's easier to parse, and a front end tools can be written for modders. But it's as you wished :D

This said, I forgot to say that there is hash collision (the hash used by TVH fields is just a for c in s : h = h<<1 + c ). For A0C00Y00 my hash resolution array use _off_ (TVH 2ee), Template's one use _inn_ (TVH:2ee).

Edit : for the missing timers, they are definitely not defined in QBN timers section. Will look to Opcode one ( can it be dynamically allocated? it seems in contradiction with QBN file format...). Maybe hacks or workaround made for DFU ?

User avatar
pango
Posts: 789
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: 0.7.2: Quest timings

Post by pango » Sat Jan 05, 2019 10:45 am

Elenwel wrote:
Sat Jan 05, 2019 9:00 am
I'm not a big fan of "natural language" representation. Even if I'm pretty confident with my current assumptions, those remain assumptions. For example I'm pretty sure that time min an max are just dropped out for destination based timer, but I chose to keep it in my current format. To avoid another global quest rewrite if someone else come with a better understanding.
Then we can decide not to interpret the base interval when some travel time is present, and deprecate their combined use.
But after converting clocks in all decompiled quests, they really seems to fit nicely inside their quests, so I'm pretty confident you got lot of stuff right; Some bits of flags seem to still have some mystery to them, so I left those as-is.

The main problem I had is that I think we didn't use quests patched identically, so I had to adapt some stuff. Several quests have been "fixed" years ago by the community, replacing some clocks with fixed durations or fixed intervals (which makes sense since clocks were poorly understood before your work); so ideally some of those fixes could be reverted/improved... with a lot more testing time.
Elenwel wrote:
Sat Jan 05, 2019 9:00 am
Ideally, I would have chosen a JSON or XML like representation for quest file, it's easier to parse, and a front end tools can be written for modders. But it's as you wished :D
That's a very interesting option, but I didn't get to choose the format, and I won't change it on my own today.
Elenwel wrote:
Sat Jan 05, 2019 9:00 am
This said, I forgot to say that there is hash collision (the hash used by TVH fields is just a for c in s : h = h<<1 + c ). For A0C00Y00 my hash resolution array use _off_ (TVH 2ee), Template's one use _inn_ (TVH:2ee).
Yes, I suspected something like that, thanks for confirming.
Elenwel wrote:
Sat Jan 05, 2019 9:00 am
Edit : for the missing timers, they are definitely not defined in QBN timers section. Will look to Opcode one ( can it be dynamically allocated? it seems in contradiction with QBN file format...). Maybe hacks or workaround made for DFU ?
I think Interkarma mostly used community patches, but after checking git logs, some changes have been made to the quests indeed (including some recently for vampirism). Both could explain discrepancies, if you used official quest files (?)
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

User avatar
Interkarma
Posts: 4005
Joined: Sun Mar 22, 2015 1:51 am

Re: 0.7.2: Quest timings

Post by Interkarma » Sat Jan 05, 2019 12:34 pm

I always regard "perfect as the enemy of good". And while I'm definitely interested in improving timings, there are multiple ways to skin that cat by this point. Please consider if the investment in time to re-export and re-integrate timers actually provides any substantial benefits over just selectively revising scripts in situ. The new timer approach likely doesn't need to be applied universally. Most quests work just fine or require only light edits for compatibility. Don't get bogged down chasing perfection at the expense of good. :)

Regarding formats such as JSON or XML, these are more suitable for storage than actually scripting quests. I selected TEMPLATE as it was already the defacto standard for Daggerfall quest scripts. It's time-tested, well documented, and easy for non-programmers to author quests. I think writers in particular adapt well to the "natural language" and task block structure of TEMPLATE. While far from perfect, I don't think we would have the proliferation of custom quests we have now from community members such as Jay and Kamer were it not for the adoption of TEMPLATE. I stand by that choice completely, and feel it has already proven to be the correct decision.

User avatar
TheLacus
Posts: 760
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Re: 0.7.2: Quest timings

Post by TheLacus » Sat Jan 05, 2019 1:05 pm

Elenwel wrote:
Sat Jan 05, 2019 9:00 am
[...] and a front end tools can be written for modders. But it's as you wished :D
I'm actually already working on a extension for Visual Studio Code that adds support for TEMPLATE. Jay_H has been using it and provided positive feedback. :)
If you are interested in creating mods for Daggerfall Unity you can find the documentation here.

User avatar
Interkarma
Posts: 4005
Joined: Sun Mar 22, 2015 1:51 am

Re: 0.7.2: Quest timings

Post by Interkarma » Sat Jan 05, 2019 4:20 pm

TheLacus wrote:
Sat Jan 05, 2019 1:05 pm
I'm actually already working on a extension for Visual Studio Code that adds support for TEMPLATE. Jay_H has been using it and provided positive feedback. :)
Can confirm this works brilliantly. :) Combined with live compilation of quests at runtime, integrated quest debugger, and console commands to start/stop quests, the game itself with VS Code is basically a quest authoring IDE at this point.

Elenwel
Posts: 5
Joined: Tue Jan 01, 2019 5:14 pm

Re: 0.7.2: Quest timings

Post by Elenwel » Sun Jan 06, 2019 4:56 pm

Yeah, as I work my way into quest script parsing I stumble upon Template Task representation. It's quite clever to transform the omnipresent 'if state then do something' into task block.

For timers, it's a you wished. I had a few bored days around new year eve, so I look into timers, that's all. :)

Edit : For completeness, my current flags understanding (I will illustrate with state over time for each timer, with a timer expiring on 4th tick):
  • x001 : default normal timer (on expiration state will be set to 1, it remains 1, states over time : 0 0 0 1 1 1)
  • x002: state will reset after next script loop ( states over time : 0 0 0 1 0 0 )
  • x004: Flip Flop timer (seen with 8 flag)
  • x008: Timer will auto restart:
    • x009 : this value happen a lot, I do not understand its meaning, maybe if state is set to 0 by script ?
    • x00A: maybe used to trigger event a fixed interval, may produce something like this 0 0 0 1 0 0 1 0 0 ...
    • x00C: Flip-flop : 0 0 0 1 1 1 0 0 0 1 1 1 ...
  • x010: Duration * 2 (and back flag)
internal flags :
  • x040: Timer is running
  • x080: Timer state
  • x100: link1 type
  • x200: link2 type
  • x400: Initialisation (link resolution) has been done
The extra week seems to be related to target location type (if uesp description of MAPD structure is correct), player is given a extra week (or two) if location is a dungeon (or in a dungeon).

All of this needs to be verified against classical Daggerfall, cause it does not seems to fit well with current quests scripts...
Last edited by Elenwel on Sun Jan 06, 2019 6:16 pm, edited 1 time in total.

User avatar
pango
Posts: 789
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: 0.7.2: Quest timings

Post by pango » Sun Jan 06, 2019 5:15 pm

Same here, the modified quests exist on a branch now; I even started modifying the parser, it looks like it could be made to work. It's currently stashed, maybe we'll get back to that if we want quest timers closer to original or need more expressive ones.

I have the feeling that something was achieved though, if anything we understand the quests better; Even if we don't jump to a brand new clock implementation right away, we'll be more informed if we need to make changes to the existing implementation (say, know when travel time is probably involved). And we have Elenwel to thank, one more time, for that.
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

Post Reply