Quantcast
Channel: Excursions into the Unknown » Assets
Viewing all articles
Browse latest Browse all 2

Game Assets: Processors

0
0

In line with my previous post about Ampere, I thought I’d share some of the scripts I’m using to build various content types for my game.

Textures

I use the DirectXTex library’s texconv tool to handle my texture processing. I’m considering moving to using my own custom format that I can serialize more efficiently, but for now DDS works fine and supports the features I need. I modified my copy of texconv a bit to be able to specify an output name as opposed to just an output directory. Here’s what the build rule looks like in my build script:

// convert textures to DDS
Build("*.texture")
    .Run("texconv.exe", "-nologo -ft DDS -pmalpha -f R8G8B8A8_UNORM_SRGB -o $(TempName).dds $(Input[0])", "$(TempName).dds")
    .From("$1.png");

This specifies that all requests for texture assets be built using texconv with DDS format, premultiplied alpha (pmalpha), output in 32-bit full color format. The output file is given in a temp directory, and the source data is a PNG of the same name as the texture asset. The final parameter to Run() specifies where the output of the tool will be, so that it can continue running the asset through the pipeline.

Materials

In my engine, I consider materials to be a collection of shaders and render states, and possibly artist-defined constant buffers. I author them as json files, and use a content processor to convert them to serialized protocol buffers. Using the excellent protobuf-net library, I can easily load this efficiently at runtime with one line of code.

// parse materials into binary blob
Build("*.material")
    .Using(MaterialProcessor.Run)
    .From("$1.json");

An example material might look something like this:

{
    "VertexShader": {
        "Name": "Common/sprite",
        "EntryPoint": "VS",
        "Profile": "vs_4_0_level_9_1"
    },
    "PixelShader": {
        "Name": "Common/sprite",
        "EntryPoint": "PS",
        "Profile": "ps_4_0_level_9_1"
    },
    "BlendState": {
        "BlendEnable": "true",
        "SourceBlend": "One",
        "DestinationBlend": "InverseSourceAlpha"
    }
}

The code to process this is surprisingly tiny and free of boring member-wise serialization code, thanks to C# reflection capabilities.

public static Stream Run(BuildInstance instance, Stream stream)
{
    var material = JsonSerializer.Deserialize<Material>(stream);

    // compile shaders
    material.VertexShaderAsset = CompileShaderNode(instance, material.VertexShaderNode);
    material.PixelShaderAsset = CompileShaderNode(instance, material.PixelShaderNode);

    var output = new MemoryStream();
    ProtoBufSerializer.Serialize(output, material);

    return output;
}

static string CompileShaderNode(BuildInstance instance, ShaderNode node)
{
    if (node == null)
        return null;

    // build up the asset name (needs to match the format in the build script)
    var name = string.Format("{0}.{1}.{2}.shader", node.Name, node.EntryPoint, node.Profile);
    instance.Start(name);

    return name;
}

One interesting thing to note is that the helper method kicks off a further asset build for referred shaders (using instance.Start()), forming the name of the asset through the shader name, entry point, and profile. We can see the corresponding build rule for such a shader:

// compiling shaders
Build("*.*.*.shader")
    .Using(ShaderProcessor.Run)
    .Run("%WinSdkDir%/fxc.exe", "/nologo /E $2 /T $3 /Zi /Od /I Common /Fd $(TempName).pdb /Fo $(TempName).fxo $(Input[0])", "$(TempName).fxo")
    .From("$1.hlsl");

Here we capture the three dotted portions of the asset name to pull the necessary parameters to compile the shader. After running the shader source file through FXC with the desired options, we then pass the compiled shader binary into a ShaderProcessor method which uses D3D shader reflection to pull out various bits of the shader needed for the engine, validates various things, and then serializes the whole chunk into protobuf format.

Packages

To avoid having to specify each asset we want built in the build script itself, I try to have most of my assets referenced by others in a dependency chain that gets discovered during build time. The shader compile mentioned above is one such instance. Another is the idea of a “package” asset, which is simply a list of assets that should be built and referred to as a logical group. This can be used by the game to load a whole package of assets at once, say for starting a specific level.

My build script uses a simple directory enumeration to find package files and kick off builds for them:

// kick off builds for each package in subdirectories
foreach (var directory in Directory.EnumerateDirectories(Env.InputPath))
{
    if (File.Exists(Path.Combine(directory, "package.json")))
        Start(string.Format("{0}/package", directory.Substring(Env.InputPath.Length)));
}

 Script References

Listing assets in a package file is still not the ideal solution. In most cases we’d just like to list a few top-level scripts, and have any assets needed by the scripts to get pulled into the build automatically. My script processor does just that. Since my game scripts are written in C#, I can use the new Roslyn API to parse the code and look for specific method calls to mark asset references.

Thus, I can search for calls to AssetRef(“foo”) and automatically include “foo” in the content build. This allows specifying assets in exactly one place and ensures that they end up in the final game data. Figuring out how to make Roslyn do this wasn’t entirely trivial, given the lack of documentation, but the final code ends up fairly short due to the nice API they’ve set up. In case you’re interested, the process involves writing a syntax walker and visiting the parsed tree:

var tree = SyntaxTree.ParseText(source, "", new ParseOptions(CompatibilityMode.None, LanguageVersion.CSharp4, true, SourceCodeKind.Interactive));
var root = new ScriptSyntaxWalker(instance).Visit(tree.GetRoot());

class ScriptSyntaxWalker : SyntaxRewriter
{
    BuildInstance instance;

    public ScriptSyntaxWalker(BuildInstance instance)
    {
        this.instance = instance;
    }

    public override SyntaxNode VisitInvocationExpression(InvocationExpressionSyntax node)
    {
        if (node.Expression.GetText().ToString().Contains("AssetRef"))
        {
            // get the asset we're trying to reference
            var arg = node.ArgumentList.Arguments.FirstOrDefault();
            if (arg == null || arg.Expression.Kind != SyntaxKind.StringLiteralExpression)
                instance.Log(LogLevel.Warning, "Script '{0}' has an invalid asset reference.", instance.OutputName);
            else
            {
                var literal = ((LiteralExpressionSyntax)arg.Expression).Token.ValueText;
                instance.Start(literal);

                // replace this node with just the name
                return arg.Expression;
            }
        }

        return base.VisitInvocationExpression(node);
    }
}

That’s a quick overview of some of the asset types used in my game. I quite like the system I have going now; it’s very flexible and requires a relatively small amount of code to get things into the format I need them to be in. The use of small optimized runtime formats will definitely help load times and reduce memory usage once I start pulling more assets into my game.


Viewing all articles
Browse latest Browse all 2

Latest Images

Trending Articles





Latest Images