Modding Tutorials: Asset-Injection

Discuss modding questions and implementation details.
Post Reply
User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Modding Tutorials: Asset-Injection

Post by TheLacus » Sat Feb 04, 2017 12:50 am

Read the updated documentation on Daggerfall Workshop website.
Daggerfall Unity provides a modding system with an integrated asset injection component. This allows artist to import meshes, textures and media files without worrying about technical details, as well as provide to the user a stable interface with a common implementation and a load order, preventing as many incompatibilities as possible. The most updated documentation is found on the main website of Daggerfall Workshop (see link above) but here are provided tutorials and suggestions. Please note that there may be some outdated content since Daggerfall Unity is still in active development. If you have any issue, please check that these docs doesn't already cover it, otherwise fell free to make a new post.

Terminology
  • Asset Bundle: a .dfmod package which includes various kind of assets. It's created with the help of the Unity Editor and installed in the StreamingAssets/Mods folder. Benefits of load order and can be updated/removed without affectings other mods.
  • Loose Files: Individual files, with a predefined format, installed inside subfolders of StreamingAssets. They override any files inside asset bundles.
  • Mod Builder: A graphical tool for the Unity Editor which allows to easily create asset bundles.
  • Prefab: A container for Unity assets such as models, materials, particles etc. which can be instantiated in game.
Features
  • Replace textures, sprites and UI images.
  • Replace models and billboards with a prefab.
  • Replace billboard trees with 3d meshes.
  • Replace midi songs with other midi files.
  • Replace sound and midi songs with a digital format version.
  • Replace cutscene videos.
Prerequisites
If you want to work with 3d models you need the free Unity Editor (at the time of writing the game uses v5.5.0f3) and a local clone of Daggerfall Unity repository. Alternatively you can install only Daggerfall Tools for Unity (dftfu) separately. The Unity Editor is required to build mods, so you'll need it if you want to release .dfmod bundles. Dftfu can be used to explore all of Daggerfall’s 3D models, cities, and dungeons but you can also use Daggerfall Modelling, while Daggerfall Imaging 2 is required to export original textures.
Last edited by TheLacus on Tue Mar 27, 2018 5:27 pm, edited 11 times in total.

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Textures

Post by TheLacus » Sat Feb 04, 2017 12:50 am

Materials
This tutorial explain how to quickly import a material, meaning a set of textures that simuate a physical surface, to be applied to a Daggerfall mesh. For this purpose we need, in addition to our favourite image editor, a local installation of Daggerfall Imaging 2.

  1. Find a texture
    Open Daggerfall Imaging 2 and set the Texture category. This will show a list of archives, which correspond to TEXTURE.XXX files from original Daggerfall Installation. Each archive contains a number of records, which are individual textures.
    Daggerfall Imaging 2.jpg
    Daggerfall Imaging 2.jpg (79.61 KiB) Viewed 6431 times
  2. Import a material
    1. Loose files
      Once we have archive and record, we can replace this specific combination with our texture, named archive_record-0.png (for example 004_1-0.png)
      and placed inside StreamingAssets/Textures folder. Launching the game is now enough to inject our texture and override the original image we have previously found in Daggerfall Imaging 2.

      At this point we can improve our materials adding textures maps such as normal, emission and metallic gloss. These textures are individuated from a tag: for example archive_record-0_Emission.png, and so on. You can find a list of all the supported tags in the documentation.

      It's worth noting that we can tweak metallicgloss properties with an xml called archive_record-0.xml. Such file is formatted like this:

      Code: Select all

      <?xml version="1.0"?>
      <properties>
      	<metallic>0.5</metallic>
      	<smoothness>0.5</smoothness>
      </properties>
      
    2. Mods
      When we import a texture with unity we can edit a few options. The most important ones are mipmaps, which should be enabled for all textures which are not used in a 2d UI, and alpha is transparency, which is self explanatory. A right click inside the project explorer allows us to create an empty material, which we are going to rename archive_record-frame.mat and fill with the albedo and other optional maps. Is also worth to set metallic and smoothness values to make the material physically realistic.

      Image
    Image
    Buildings with a custom albedo texture. Since they have windows, an emission map is also required.
Billboards
  1. Find a texture
    Sprite textures can be found with Daggerfall Imaging 2 within the Texture category. If you want to find which texture is used for a specific billboard, you can use Daggerfall Tools for Unity with Batch Billboards disabled and load a block with the importer. Billboard textures can be animated, so a record within an archive may have more than one frame.
  2. Import the texture
    1. Loose files
      Place a texture named archive_record-frame.png inside StreamingAssets/Textures folder. If the billboard is animated, be sure to import all frames.
      If the billboard emits light, an emission map can also be provided (archive_record-frame_Emission.png).
    2. Mods
      Billboard textures can also be imported with a .dfmod pack. Import your textures with Unity and name them archive_record-frame and archive_record-frame_Emission. Enable alpha transparency from the inspector and tweak other import options as required. Default import mode with mipmaps enabled should be fine in most cases.

      Open Daggeral Workshop tools>Mod builder, add all your textures and build to create a new .dfmod pack ready to be imported from the StreamingAssets/Mods folder.
    Image
    An animated torch with an emission map.
  3. (optional) Set the UV
    For a simple texture swap, only texture files are needed. They will have the same size as vanilla, regardless of resolution. The downside is that they need to have the same proportions as the original textures or they'll look stretched.
    But if you want to follow the power of two rule, you can do so if you take care of telling the game how the texture should be displayed on the billboard mesh. Specifically, you need to provide an xml file with the position of the lower-left vertex on the uv map. All frames share the same file.

    Image

    Archive_Record-0.xml

    Code: Select all

    <?xml version="1.0"?>
    <info>
    	<uvX>0.3</uvX>
    	<uvY>0.1</uvY>
    </info>
    Tipically, you want to avoid wasting texture space, so you only need to write the x or y value and the other will be set to the default value of zero.
  4. (optional) Set a scale
    If you want to customize the size of the mesh you can do so with the xml file. All frames share the same file.
    For example this has been used for the Remove spider mod, which replaces horizontal spiders with vertical beards avoiding innatural streching.

    Archive_Record-0.xml

    Code: Select all

    <?xml version="1.0"?>
    <info>
    	<scaleX>1</scaleX>
    	<scaleY>1</scaleY>
    </info>
    (1:1) means the same size as vanilla, (1:2) has a double height, (0.5:1) is half as wide. If you want to customize only one of the values, you can omit the other one.
Last edited by TheLacus on Sat Apr 28, 2018 2:35 pm, edited 15 times in total.

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Textures

Post by TheLacus » Sat Feb 04, 2017 12:51 am

UI Images
  1. Find a texture
    Daggerfall uses images stored as .IMG, .CIF and .RCI packages for the hud, inventory and all the UI windows which are found in game. The original textures can be exported with the help of Daggerfall Imaging 2. Daggerfall Unity also include its own additional graphics for the main menu and its windows.
  2. Import the texture
    When we import a replacement texture, we follows a nomenclature that allows them to be retrieved. Img files have no animations so the name is the same you see in Daggerfall Imaging (ex: MAP100I0.IMG). Cif and Rci can have several frames so we use filename_record-frame (ex: INVE16I0.CIF_3-0). Weapons used by the player have several variations for different kinds of metal. Vanilla textures use a color tint to achieve this; when importing custom textures, we use different images with tags. You can find an updated list in the documentation.
    1. Loose files
      Textures can be imported directly from StreamingAssets\Textures\Img and StreamingAssets\Textures\CifRci. Use png format with ".png" extension for Mac and Linux support (lowercase and uppercase are seen as different strings on these platforms). Examples: StreamingAssets\Textures\Img\MAP100I0.IMG.png, StreamingAssets\Textures\CifRci\INVE16I0.CIF_3-0.png.
    2. Mods
      UI images can also be packed inside a dfmod bundle. Place your textures inside as subfolder of Assets and Unity will import them. Suggested Import options are Texture Type: GUI, mipmaps: disabled. if required, enable Alpha is Transparency.
  3. (optional) Set the size
    Certain UI images need additional informations. These are provided using .xml files, each one with the same name as the texture file it refers to and placed in the same folder. width and height parameters affect the size on screen. Here are provided the defaults values, but it's possible to customize them.
    • HUD Compass
      COMPASS.IMG.xml

      Code: Select all

      <?xml version="1.0"?>
      <info>
      	<width>322</width>
      	<height>14</height>
      </info>
      COMPBOX.IMG.xml

      Code: Select all

      <?xml version="1.0"?>
      <info>
      	<width>69</width>
      	<height>17</height>
      </info>
    • Crosshair
      Crosshair.png and Crosshair.xml in StreamingAssets\Textures

      Code: Select all

      <?xml version="1.0"?>
      <info>
      	<width>22</width>
      	<height>22</height>
      </info>
Save Manager
It is possible to customize the window interface used to load saves, placing custom textures inside StreamingAssets\Textures.

Image

While it is not required for swapping texture images, it is possible to use xml files to customize the labes of buttons.

Image
  • Change text color

    Code: Select all

    <?xml version="1.0"?>
    <info>
    	<customtext>true</customtext>
    	<r>0.8</r>
    	<g>0.1</g>
    	<b>0.1</b>
    	<a>1.0</a>
    </info>
    An omitted alpha value is considered as full opacity.
  • Remove text (to draw text on texture)

    Code: Select all

    <?xml version="1.0"?>
    <info>
    	<customtext>notext</customtext>
    </info>
Last edited by TheLacus on Tue Jun 12, 2018 8:23 pm, edited 21 times in total.

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Models and Flats

Post by TheLacus » Sat Feb 04, 2017 12:51 am

Models
  1. Find the model ID
    When you open Daggerfall Modelling, you can see a list of all the meshes used by Daggerfall. Each one of them has a model ID. Is a good idea to export the mesh as a reference for size and origin; note that Daggerfall scale is forty times Daggerfall Unity scale (0.025).
    Daggerfall_Modelling_ModelID.jpg
    Daggerfall_Modelling_ModelID.jpg (20.35 KiB) Viewed 6198 times
  2. Import the model
    Before exporting a model from your modelling software, remember to set Y as the up axys; if you use Blender there is a plugin available. Several formats are supported, but for the purpose of this tutorial i will refer to a .fbx file.

    Image Image
    Export settings in Blender.

    Once you have placed this file in a subfolder of Assets (Game/Addons/MyModelsPack for example), unity will import it and create a folder called Materials, containing one .mat file for every material included in the fbx. If you assign textures to them, they will be shown on the model. At this point your model (from now on: GameObject) is ready, but you can further customize it dragging it in the scene view. You can tweak it assigning different materials and adding components such as child models (useful if you need to separate a mesh with more than 65,534 vertices), particles, lights and anything else you can find in the Inspector, under AddComponent. Once you're done, drag it back in the project explorer to create a prefab. Prefabs are also useful if you find the same model under multiple IDs (which happens a lot...). It's a wise idea to have different prefabs that share the same assets, rather than making actual copies.


    ImageImage
  3. Build the mod
    Now it's time to build your new mod. Open Daggerfall Workshop>Mod Builder, set name, author and other fields, then proceed to select and add your gameobject, named modelID.prefab (or modeID.fbx). Do not select prefab components manually because they are added automatically. If you do, they will be included as individual assets (as opposed to being children of the prefab), which will increase the mod size without any benefit (for the purpose of what we are doing). Once you obtained a .dfmod file, place it in StreamingAssets/Mods and launch the game to see your model in game.

    Image
  4. Make variations
    Daggerfall reuse the same mesh with different textures to achieve models for different locations or season. Instead of limiting ourselves to textures, we can provide different prefabs for this purpose. A prefab can use the same mesh with different materials, or use entirely different meshes to increase variety. This feature is provided through tags; refer to the documentation for an updated list.
Last edited by TheLacus on Tue Mar 27, 2018 8:14 pm, edited 10 times in total.

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Models and Flats

Post by TheLacus » Sat Feb 04, 2017 12:52 am

Flats
Daggerfall Unity uses billboards (sprites) for NPCs, enemies and vegetation. In this tutorial we are going to replace the with actual 3d models.
  1. Find archive and record
    There are two ways to identify a sprite texture. Daggerfall Imaging 2 list all of them under the Textures category, with all other textures for meshes. Daggerfall Tools for Unity, used within the Unity Editor, allow you to find the name of a billboard at a specific location, provided that Billboard Batch is disabled before loading the scene in the scene view.

    Once we have archive and record we can proceed; the last number is the animation frame, which is not required because animations must be provided by the model we are going to import.
  2. Import a model

    Image
    Replacing sprites with 3d models.

    Refer to the tutorial for models above and name your gameobject archive_record. Custom flats prefabs are placed in game assuming the origin of the model is at the base, where they touch the ground or the surface they are positioned on, so place it in this position.

    If the model is supposed to collide with the player, check Generate colliders in the mesh inspector inside the Unity editor.
  3. Tweak the gameobject
    Since sprites rotate to always face the player, when we replace them with models we don't have a rotation value for an exact placement. In most cases this is not a problem, as they are used for vegetation and random props, but there are a few exception where there is ony one correct position, for example wall torches.
    In this case we can use a script to place the model correctly in all circumstances. Select your prefab, click Add Component in the Inspector and search FaceWall.cs. Refer to the documentation for details.

    Image
  4. Build the mod
    Refer to the tutorial for models above.

    Image
    Mesh, point light and particles emitter. This prototype by Uncanny_Valley does a great job showing what's possible to do.
Last edited by TheLacus on Wed Mar 28, 2018 10:24 am, edited 11 times in total.

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Sound and Movies

Post by TheLacus » Sat Feb 04, 2017 12:52 am

Sound
  1. Find sound name
    Sound names are listed here (environment) and here (songs).
  2. Import sound file
    1. Loose files
      Export your sound file in wav format and your song file in ogg format. Place it inside StreamingAssets/Sound.
    2. Mods
      Import your sound file with the Unity Editor. There are a few options you can set; the most important ones is Load Type (uncompressed, compressed or streamed). You can find an explanation of all settings here.
      Image
  3. Test song
    Run the game and input in console tdbg. Current song and playback position are shown on the top left corner.
Movies
TBA
Last edited by TheLacus on Thu Jun 14, 2018 9:42 am, edited 9 times in total.

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Documentation

Post by TheLacus » Sat Feb 04, 2017 12:53 am

Useful Links
Some useful pages from the Unity documentation.
Last edited by TheLacus on Tue Mar 27, 2018 8:49 pm, edited 3 times in total.

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Re: Asset-injection

Post by TheLacus » Sat Feb 04, 2017 12:53 am

Known Issues
Core modding features that are not implemented, have issues or require some polishing.

Textures
  • Wandering NPC: material issue with DXT5 (compression enabled in settings) and mipmaps, maybe related to PO4 requirements.
  • Some textures for UI are still not supported because windows were made before modding support was in place and they require important changes. This is a delicate matter because it necessarily affects core.
    • World map (TRAV0I00.IMG)
    • Local maps (FMAPxxxx.IMG)
    • Paperdoll (BODY00I0.IMG)
    • Archives 233-234, 235-242, 245-252, 254
Proposed Features
The following is a list of features that may be implemented in the next future. I'm happy to receive suggestions from the people here who made mods or used them, specifically on aspects that you think deserve some smoothing.
If you have new features to propose also feel free to do it but please try to be concrete. I'm not asking for mod ideas, i'm asking for feedback on the modding system. Italic for advanced features which may be a long term target or even something to delegate to a mod.

Mod System
  • Settings backend should be pretty stable at this point, especially in game. Editor window could benefits of some minor usability tweaks.
  • Lypyl once talked about his intention to allow toggle and settings changes of running mods. This can be implemented with callbacks and, with some addition, the mod manager window on start menu can also be used in game for this purpose (with a button on classic pause menu or only via console?). Mods which contains only assets would not implement these callbacks but they should also be toggable, maybe followed by caches refresh.

Models
  • Custom door positions (i have some prototypes in a branch but i don't think this is a priority).
  • Models are the only asset type which can't be released as loose files. It's possible to add support for .obj import from disk but i'm not sure if this is a worthy feature, loading time would surely be bigger than prefab import. I'd listen to community feedback here.
  • Import meshes without textures? Not sure how much of original data must be loaded to retrieve texture IDs. Can be helpful to fix bad UV on vanilla assets without affecting texture replacement.
  • Support for 3d first person weapons.
  • Support for 3d animated NPCS and enemies.
Textures
  • Additional textures on materials (ie. heightmap and occlusion) can be easily supported if there is a request from modders. This is necessary because they have to be enabled in shader and a name tag muste be defined for loose files.
  • Billboard have a 1:1 replacement of albedo and emission. Enabling other textures may require changes in core. There can be some initial confusion with billboards due to different implementations (ie. trees use a 4096x atlas, npcs individual textures with support for higher resolution). Documentation should explain all this but new features should also be implemented as organically as possible.
Sound
  • Songs could be streamed to try avoid lag on track switch. Had issues with Unity 5.5, will try again with 2018. This affects only loose files, options for audioclip from mods are set by modder.
Movies
  • Current implementation draws a movietexture as a panel background. New Unity versions (5.6+) introduce a new way to play videos (VideoPlayer) which draws on a rendertexture (or set directly on near/far plane of camera) and ought to be more performant.
Last edited by TheLacus on Tue Jun 12, 2018 8:28 pm, edited 13 times in total.

User avatar
kingOfWyrms
Posts: 44
Joined: Fri Aug 07, 2015 3:46 am

Re: Mesh and Texture replacement - guide

Post by kingOfWyrms » Sat Feb 04, 2017 12:32 pm

TheLacus wrote:The name of the Mod must be the id of the model (modelID or Archive_record).
Once we have built the Mod we can find three folders, one for each operating system. What's important it's the modelID.dfmod file.
Does this mean we'll need to have a separate asset pack for each item we're replacing?

User avatar
TheLacus
Posts: 596
Joined: Wed Sep 14, 2016 6:22 pm
Contact:

Re: Mesh and Texture replacement - guide

Post by TheLacus » Sat Feb 04, 2017 7:15 pm

kingOfWyrms wrote:
TheLacus wrote:The name of the Mod must be the id of the model (modelID or Archive_record).
Once we have built the Mod we can find three folders, one for each operating system. What's important it's the modelID.dfmod file.
Does this mean we'll need to have a separate asset pack for each item we're replacing?
Yes, for now. It's my intention to integrate it more with the mod system and support importing models from mods, with load order and all, but this require still some work.
If you are interested in creating mods for Daggerfall Unity you can find the documentation here.

Post Reply