Cogs Material Format

Specification of json format used for material Cogs definition.

Cogs Material Format

Introduction

Basic overview

The materials describe how geometry vertex data is transformed and end up as SurfaceOut (see Common.hlsl) values. Then, an engine permutation(not part of the material) describes how SurfaceOut values result in rendertarget values.

A material has a set of permutations (not to be confused with engine permutations). Only one permutation can be active at a time. In addition, a material has a set of variants that can be set to values independent of the permutation. Finally, a material can inherit from other materials. All materials inherit from MaterialBase.

A material template is an abstract material that cannot be instanced directly, only inherited, and does not have permutations.

Debugging

Shaders go through a lot of compositing and transformation when processed by the material system, and when something goes wrong, error messages from the graphics API reference the source transformed source.

Two cogs-variables let you peek into this:

If effects.logShaderSource is true, and the shader fails to compile, the transformed shader source is dumped to the log at debug loglevel.
WebGL2/ES3 backend: If effects.logShaderInfo is true, info about attributes and uniforms as well as shader source is always dumper to the log at debug loglevel.

Connection to MaterialComponent/MaterialSystem

The cogs material are resources. An application can directly manage these, creating new resources and setting properties. The renderComponent binds an material instance to an entity.

As a simplification, some entities has a MaterialComponent managed by the MaterialSystem. This expose a set of "usual" material properties and manages a set of DefaultMaterial-instances behind the scenes.

You have to choose whether you want to manage the material resource for an entity yourself, or let the material system do it for you.

Do not set a MaterialInstance resource if the entity has a MaterialComponent. Choose another entity type or remove the MaterialComponent.

How different rendering APIs are handled

Cogs use hlsl files for shaders and DirectX semantics. For desktop APIs beside DirectX, the shaders get translated by Cogs.Rendering to the APIs native shader language.

For OpenGLES3 (i.e. webgl), shaders are not translated. Thus, the material author needs to provide glsl translations. The material system replaces the .hlsl suffix with es30.glsl depending on backend, and use a dedicated shader generator.

Materials and material instances

A material instance is an actual instance of material, i.e. holds the material data. A material instance has a material, which defines the data. Roughly, a material defines a property diffuseColor, but the material instance sets diffuseColor to the value of red.

Relevant files in the code base

serialization/MaterialReader.cpp: Parses material files, builds material structures, handles inheritance.
resources/MaterialManager.cpp: Handles runtime instantiation of a material into a shader, that is, applying permutations and variations and checking preconditions.
resources/ShaderBuilder.cpp: Transforms a configured material and shaders into full hlsl shaders passed to Cogs.Rendering. ShaderBuilderES3.cpp does the same for OpenGLES3 respectively.
renderer/RenderMaterial.cpp: Manages effect bindings, mapping from material instance, stream and engine permutation to an actual concrete shader with constant buffer binding points.
renderer/RenderMaterialInstance.cpp: Manages buffer/texture properties tied to a render material.

JSON grammar

Top level structure

The basic .material file structure is an JSON object with either a Material or MaterialTemplate key.

{
    "Material": { },
    // or
    "MaterialTemplate": {}
}

Contains exactly one of:

Material: Combined object with keys from both Material and material template metadata and Material effect definition.
MaterialTemplate: Combined object with keys from both Material and material template metadata and Material effect definition.

Material and material template metadata

Materials and material templates are similar. A material is assumed to be instantiated and have permutations, an material template is assumed to only be used for inheritance and do not have permutations.

{
    "name": "",
    "flags": "",
    "inherits": "",
    "requires": ,
    "variants": ,
    "options": {},
    "properties": {},
    "sharedProperties": {},
    "permutations": {},
    "enginePermutations": []
}

name: String with name of material.
flags: or string array, currently no flags are defined.
inherits: String with name of material to inherit
require or requires: specifies variant requirements, see Require statements.
variants: specifies variants, see Material variants.
options: Object with string values defining, see Material options.
properties: Object specifying material per-instance properties, see Material properties.
sharedProperties: Object specifying material properties that are shared between all material instances, see Material properties.
permutations: Object with permutation definitions, see Material permutations.
enginePermutations: Array with strings, specifies which Engine permutations material is valid for. The default enabled are Forward, Deferred, Shadow and Transparent.

Material effect definition

{
    "vertexFunction": {
      "file": "xyzVS.hlsl"
    },
    "surfaceInterface": {
      "foo": "float4",
      "facing": "SV_IsFrontFace"
    },
    "surfaceFunction": {
      "file": "xyzPS.hlsl",
    }
  }

Shader stage interfaces, see interface definition:
- vertexInterface: Specifies input to the vertex shader, that is the mesh streams. Variants in MaterialBase that add to the vertex interface gets automatically enabled/disabled depending on mesh contents, so usually it is not necessary to explicitly specify this.
- hullInterface: Input to the hull tessellation shader stage.
- domainInterface: Input to the domain tessellation shader stage.
- geometryInterface: Input to the geometry shader stage.
- surfaceInterface: Input to the surface shader (first part of pixel shader) stage.
Shader stage definitions, see function definition:
- vertexFunction: Specifies vertex shader stage.
- hullFunction: Specifies hull tessellation shader stage.
- domainFunction: Specifies domain tessellation shader stage.
- geometryFunction: Specifies geometry shader stage.
- surfaceFunction: Specifies surface shader (first part of pixel shader) stage.
definitions or defines: Object with string values that will be passed as preprocessor defines to all shaders.

Shader interface definition

Object that specifies the input to a shader stage. Each key is a input variable name visible in the shader, the value specifies how this binds to the surrounding stages:

{
    "ShaderInputNameA": "float3",
    "ShaderInputNameB": ["float3", "TEXCOORD0" ],
    "ShaderInputNameC": ["float3", "TEXCOORD1", "linear" ],
    "ShaderInputNameD": ["float3", "TEXCOORD2", ["linear", "noperspective"] ]
}

First element is type:

bool: Boolean value
float, float2, float3, float4: Floating point vector.
uint, uint2, uint3, uint4: Unsigned integer data.
int, int2, int3, int4: Integer data.
float4x4: Float 4x4 matrix.
float4[]: Array of float4, second element is assumed to be array dimension either string or value.
VFACE:
SV_IsFrontFace:
SV_InstanceID:
SV_ClipDistance:
SV_Position:

Second element, optional, holds semantic, x denotes optional slot number:

POSITIONx:
NORMALx:
COLORx:
TEXCOORDx:
TANGENTx:
INSTANCEVECTORx:
INSTANCEMATRIXx:
SV_Position: System provided.
SV_VertexID: System provided.
SV_InstanceID: System provided.
SV_ClipDistance: System provided.
VFACE: System provided.
SV_IsFrontFace: System provided.

Third element, optional, holds interpolation modifiers, can be either a string or an array of strings:

linear:
centroid:
nointerpolation:
noperspective:
sample:

Shader function definition

Object that specifies the shader snippet for a stage

{
    "file": "xyzGS.hlsl",
    "entryPoint": "myGeometryFunction",
    "attributes": {
        "primitivetype": "point",
        "maxvertexcount": "4"
    }
}

file: Path to source hlsl shader file.
entryPoint: Optional entry point for shader function.
attribute: Optional attributes, currently these for geometry shaders:
- primitivetype:
- maxvertexcount:

Require statements

Require statements require that variants have a particular value.

As a string:

"requires": "DefaultHasNormal"

or an arrayt of string:

"requires": [ "DefaultHasNormal", "DefaultHasTexCood" ]

this requires that the variant DefaultHasNormal is set etc.

As an object:

"requires": {
    "VertexStreamPosition0": "float4|float3"
}

this requires that the VertexStreamPosition0 variant either has a value float4 or float3. This is useful to set up mesh requirements for the materials, as the "VertexStramXXXXN"-variants are automatically populated with the type of mesh streams.

Material options

Material options are a JSON object with string values, handled in resources/MaterialOptions.cpp.

Recognized key-value pairs:

CullMode: Front or Back.
Transparency:
- Value On:
- Value Off:
- Value Auto:
- Value Alpha:
DrawOrder: String containing an integer.
BlendMode: String with BlendMode from MaterialOptions.h.
DepthWriteEnabled: String interpreted as bool. Enables/disables writing of depth buffer, note: requires enabled depth test.
DepthTestEnabled: String interpreted as bool. Enables/disables the depth test.
DepthTestAlwaysPass: String interpreted as bool. Depth test always passes, useful for writing depth unconditionally.
DepthBiasEnabled: String interpreted as bool. Enables/disables depth bias.
DepthBiasConstant: String interpreted as float. Sets the depth bias constant, requires enabled DepthBiasEnabled.
DepthBiasSlope: String interpreted as float. Sets depth bias slope factor, requires enabled DepthBiasEnabled.
DepthBiasClamp: String interpreted as float. Sets depth bias clamp limit, requires enabled DepthBiasEnabled.

Material properties

The properties section contains all the user-defined properties of the material, like constant buffers (sometimes referred to as material property buffers in Cogs), and textures.

Material property buffers may contain the following data types:

float
float2
float3
float4
float4[]
float4x4
uint
int
int2
int3
int4
bool

Properties are reordered by Cogs to create a suitable structure layout for use both from native code and shader code.

"properties": {
    "MyProperties": {
        "myFloatProperty": "float 1.0",
        "myColorProperty": "float3 srgb { 1, 1, 1 }",
        "myVec4Property": "float4 { 0, 1, 0, 1}",
        "myBoolProperty": "bool false"
        // Etc...
    },
    "MyTexture": "Texture2D",
    "MyCubeTexture": "TextureCube"
}

Material permutations

The permutations section contains the combinations of shader code and interfaces that together with engine shader code combines to complete runnable shaders and corresponding input/output structures.

As a shorthand, the permutations section may be dropped in favor of a single "default" permutation specified at the material scope.

Material variants

Variants are toggles for features that can be turned on or off in a material. A variant has a key, which defines a 'configuration axis' and a type that defines allowable values along this axis. The configuration value is forwarded to the shader source via a preprocessor defines, and by using #ifdef's, parts of the shader source can be omitted or included before passing it to the rendering API. Thus, variants are more fine-grained than material permutations.

Bool variants is either enabled or disabled. Example:

"MyBoolVariant": {                // Key
  "type": "bool",                 // Type
  "value": "MY_VARIANT_DEFINE",   // Shader define value
  "trigger": "...",               // Optional auto-enable by other variants.
  "surfaceInterface": { /*...*/ } // Optional data passed between shader stages
}

Here, setting materialInstance->setVariant('MyBoolVariant', true) will put #define MyBoolVariant 1 in the top of the shader source, while setting it to false will omit the define. Also shown is the optional trigger that allows a bool variant to be automatically enabled if another variant is enabled. Finally, this variant also add some data to be passed into the fragment shader stage.

Int variants allow you to pass an integer number to a define

"MyIntVariant": {
  "type": "int",
  "value": "MY_VARIANT_DEFINE",
  "defaultValue": 0
}

Here, setting materialInstance->setVariant('MyIntVariant', 42) will put #define MY_VARIANT_DEFINE 42 in the top of the shader source.

Enum variants specifies a set of defines where the one matching the set value is defined to 1, while the others are omitted. Example:

"MyEnumVariant": {
  "type": "enum",
  "values": {
    "ValueA": "PREPROCESSOR_DEFINE_FOR_A",
    "ValueB": "PREPROCESSOR_DEFINE_FOR_B",
    "ValueC": "PREPROCESSOR_DEFINE_FOR_C"
  },
  "defaultValue": "ValueA"
}

Here, setting materialInstance->setVariant('MyEnumVariant', 'ValueB') will put #define PREPROCESSOR_DEFINE_FOR_B 1 in the top of the shader source.

Format variants is similar to bool variant but contains a format:

"MyFormatVariant": {
  "type": "format",
  "value": "MY_FORMAT_VARIANT",
  "surfaceInterface": {
    "foobar": "format"
  }
}

Here, setting materialInstance->setVariant ('MyFormatVariant', 'R32G32B32_UINT') will put #define MY_VARIANT_DEFINE COGS_UINT3 in the top of the shader source and add an uint3 input to the fragment shader stage. This is mostly used in MaterialBase.material for automatic enabling from mesh stream layouts, and is used to trigger more high-level variants (like presence of normal vectors).

Material top level structure

{
    "name": "",
    "type": "",
    "properties": {

    },
    "options": {

    },
    "variants": {

    },
    "permutations": {

    },
}