Page 1 of 16

Daggerfall Unity VR with the Vive Pro

Posted: Mon May 28, 2018 3:23 am
by InconsolableCellist
So I've done a (buggy) thing, and I think I'll keep doing that thing:

Re: Daggerfall Unity VR with the Vive Pro

Posted: Mon May 28, 2018 3:48 am
by Interkarma
Hey IC! Good to see you back around these parts again. :)

That looks cool as hell, looking forward to seeing where it goes.

Re: Daggerfall Unity VR with the Vive Pro

Posted: Mon May 28, 2018 7:29 pm
by InconsolableCellist
Thanks, @Interkarma! I feel guilty I haven't contributed in a long time, and meanwhile you're killing it. What you've done with Daggerfall Unity is absolutely amazing. I'd like to give back and help improve it, and hopefully this'll be a good addition.

How are things going generally with DFU? I catch up on the blog posts every so often. One thing I don't know anything about is the mod framework. I've just been stripping things out willy nilly to make VR work, but I should do it as a mod, but I have to learn how. (meaning I"ll read ... y/modding/)

Re: Daggerfall Unity VR with the Vive Pro

Posted: Mon May 28, 2018 10:36 pm
by Interkarma
Things are going great with Daggerfall Unity! Quest system almost completely operational and magic effects are being written now. The project has attracted some wonderful contributors in the last couple of years and is building out at a fast rate. All these other game systems (guilds, commoner quests, background alliances, banking, renting tavern rooms, buying houses, guild services, etc.) have all come online while I've been in the 0.4-0.5 tracks. By the time I'm done with 0.5, we'll basically race through 0.6, 0.7, 0.8 milestones in one fell swoop. Check out the Roadmap for more.

You might have a hard time working in VR as a mod. While the modding system is plenty powerful, the needs of VR are so integrated across the whole game you're probably better off going for a fork and building in the changes you need.

I've had a bit of a play with this on my end as well (have a Vive not a Pro, so jelly) and there are a few really sticky areas.
  • Lighting - VR looks best with forward rendering and lots of AA, but DFU is deferred for all those point lights. This means you either end up with flickery lights with AA on forward or great lighting with no AA on deferred. You can use deferred with FXAA and Temporal AA in VR with varying tradeoffs. No perfect solution yet, although when I bump to Unity 2018.1 next month it might offer some more options.
  • GUI - The non-diagetic UIs in DFU are hard to port into VR. You'll need to rewire the UI back-end to render to an offscreen surface then present that on a diagetic quad the user can point to with the controller.
  • Weapons, horse, spellcasting, etc. - All need models to replace the 2D overlay animations. This is an area you might get some traction with modders in general, as nice 3D versions of hands and weapons could be used in base game as a mod also.
They're the main problems I struck in my little VR hackathon. I'm sure as things progress there would be more. But once you sorted the base game rendering, UI, selecting things, movement, and attacking the rest should fall into place relatively quickly as most of the game wouldn't need to be touched.

Good luck with it! I'm really excited to see where someone like you could take this.

Re: Daggerfall Unity VR with the Vive Pro

Posted: Mon May 28, 2018 11:06 pm
by InconsolableCellist
Ha funny you should mention those things, as I was putting those together as theories for problem points. I didn't think about the lighting (it actually looked really good from my brief testing), but modding was a concern, as was realizing that I'd have to figure out the UI situation. It seems that somewhere a top level Panel is added (uiManager maybe?). First step would be grabbing that and sticking it on a Canvas/GO parented to the controller, I think, or similar.

This could all be tackled, but it makes me extremely nervous making changes in pre-alpha without modding, as I'd then have to deal with a million merges, each of which have a high chance of seriously breaking my weird little fork. However, a fork would give me a ton of flexibility to strip out things entirely and go nuts, so I don't have to support both a 2D and VR version. Though if I do start handling both, and since you have a Vive to test (the Pro is amazing, by the way!), could these changes make it upstream?

I do believe it'd be possible to do with modding, painful as it may be. I modded in (very rudimentary) Oculus Rift support to Cities: Skylines without having access to the source code. But I'm sure that too would've broken during updates, and I didn't deal with the UI.

Re: Daggerfall Unity VR with the Vive Pro

Posted: Mon May 28, 2018 11:33 pm
by Interkarma
I'll make you a deal. If you'd like to settle on the modding system, then I'm happy to help build the support into core so it can get off the ground. My only requirement is that we don't do any major restructuring that would break core. I'm pushing towards leaving pre-alpha this year and don't want to upset the process of continuous refinement or end up in feature creep.

Let me take a look at the UI system, as I already have a few ideas around this from my earlier hacking. DagUI is a custom UI setup I built from scratch to perfectly emulate Daggerfall's little UI quirks, so I can possibly cut through some of the problems quickly due to insider knowledge. And I'll be responsible if something breaks. :)

If anybody out there is willing to create 3D models of Daggerfall's weapon animations (see WEAP*.CIF files in Daggerfall Imaging 2) this is something we can use as a mod in base game as well.

For lighting and AA, I recommend leaving deferred enabled. You can use FXAA and SSAA from the post-processing mod already. These will go a long way to improving aliasing without breaking the lighting with forward rendering.

Once you have a play with the modding system and injecting scripts/assets etc. through a .dfmod file, you should be able to work out what hooks are needed to support the VR setup in core.

My main focus will continue to be game systems, I don't want to let myself become distracted from that. So besides helping with the UI rendering, I won't be able to become that involved. That should be OK though as this is your baby. :)

Re: Daggerfall Unity VR with the Vive Pro

Posted: Mon May 28, 2018 11:51 pm
by InconsolableCellist
Indeed, I wouldn't want this highly specific and seldom used feature to influence the core to the point where it detracts from Daggerfall Unity's main mission: reimplementing a playable version of Daggerfall in Unity w/ DFTFU. I'm not sure what changes will be necessary yet, but as one example I already modified the Mod Builder to allow for recursive paths to be selected when adding assets (I thought it was a bug for a while actually).

I imagine the mod is going to do a lot of searching for existing GameObjects and replacing them with modified prefabs that have the scripts necessary to handle VR interaction. However, as long as the mod system lets me execute arbitrary code and insert prefabs, assets, and scripts, it should be possible. (I did read about that one limitation of no enums due to a compiler issue, that could interfere with Vive/Steam's stuff.)

I also ignorantly grabbed the latest and greatest Unity. DFU seems to build and run, though I discovered that one change is necessary: Edit -> Project Settings -> Player, change API Compatibility Level from .NET 2.0 Subset to .NET 2.0. Otherwise it can't find CompilerParameters I think it is.

Re: Daggerfall Unity VR with the Vive Pro

Posted: Tue May 29, 2018 12:14 am
by Interkarma
Yep, its all compatible with latest Unity versions. I've been working towards this since around Feb in preparation, but won't be pulling the trigger officially on the git version until mid next month. Setting the .NET level has been a requirement since the mod system went in, but it should clone this project setting normally if using the same Unity level as git (5.5.0f1 currently). Once you upgrade levels, this setting gets reverted for some reason.

If you need to make any changes to mod system itself, please run them past TheLacus before submitting a PR. He's the caretaker of the mod system in its current form.

Not sure how big a deal the enum thing will be in practice. Hit up some of the guys with more experience than myself (TheLacus, Hazelnut, Uncanny_Valley) as I believe they have some good ways of working around this.

I've created a fork on git to play with some of the UI stuff without touching core. If I end up too much in the weeds, I can at least report on the problems encountered. :)

Re: Daggerfall Unity VR with the Vive Pro

Posted: Tue May 29, 2018 5:41 am
by Interkarma
OK, I've pushed a really rough proof-of-concept to the ui-rendering-test branch on git. Here's the overview of how it works.

Edit: Fixing spelling errors and updating information (30-May-2018).

  • Adds a new component class to DaggerfallUI called UserInterfaceRenderTarget.
  • Adds a new child of Canvas GameObject in scene called NonDiegeticUIOutput.
  • The NonDiegeticUIOutput object basically just renders a RenderTexture to a canvas.
  • All GUI.DrawTexture() styled calls are now sent to DrawTexture() wrappers inside DaggerfallUI.Instance.RenderTarget that divert this rendering to a RenderTexture.
  • The RenderTexture is cleared on every OnGUI refresh.
Pancake Mode
  • In normal rendering, the NonDiegeticUIOutput object just draws whatever is in the TargetTexture to the canvas stretched over the screen. This works much like before where it was drawing directly to the frame buffer with GUI.DrawTexture().
Using In VR
  • First disable NonDiegeticUIOutput object. You can get a reference to this from DaggerfallUI.Instance.NonDiegeticUIOutput.
  • Setup a quad, curved panel, cube (whatever you like) to show your diegetic UI parent object to the player. Place this so they can see it comfortably. This will be used to display the RenderTexture in the VR world.
  • Set the diegetic UI object material to use DaggerfallUI.Instance.TargetTexture texture to see whatever is in the UI render texture.
  • Probably everything. I put this together in like an hour in between phone calls. Problems are to be expected. This has had a lot more work now and I'm starting to feel better about the outcome. Still have more to fix however.
  • Breaks the startup UI - make sure you're running the game scene directly.
  • There's currently no way to tell the UI system what the pointer coordinates are when aiming with the motion controller's "laser pointer".
  • There's a slight flicker when pushing/popping through window stack.
  • Some UI components are now semi-transparent based on how this works. Not worried about this just yet in PoC. Problem is with additive blend on GUI.DrawTexture(). Turns out to be not much of a problem in practice as I rarely need it.
I'm going to clean this up a bit more when I have time, please don't send any PRs to this just yet. But feedback is welcome. Let me know if this concept will be helpful for rendering the Daggerfall Unity UI in a more diegetic way. :)

Re: Daggerfall Unity VR with the Vive Pro

Posted: Tue May 29, 2018 7:13 pm
by InconsolableCellist
This is awesome! Thanks so much. I'm going to pull down that branch and dig into this shortly. This may be a stupid question, but what's diagetic? Is it diegetic? I had to look up both, and it seems it might be a misspelling of diegetic, which apparently means that the UI is actually in-game, which would certainly make sense for VR UIs (e.g., that curved quad floating out in front of you like Iron Man's HUD curved around his face).