Work in progress..
Cogs.Control is the entry point in the system.
Cogs.Runtime (Accessed using control.runtime) handles creation of entities and components. Also owns important classes like Resources and Scene. Can also load entities defined using json using load... functions.
Cogs.Scene (Accessed using control.runtime.scene) contains picking, origin, entity search and bounding box queries.
Cogs.Resources (Accessed using control.runtime.resources) handles loading of textures, models, assets. Also resource lifetime controlled here.
Cogs.Entity is the root class of all Cogs Entities. Entities are the building blocks of creating 3D graphics in Cogs.
Entities are stored in Cogs / GPU offering retained graphics. A hierarchy is offered using SceneComponent.children.
Child entities are defined with positions local to parent.
Rough grouping of Entity types:
Predefined objects: Cube, Sphere, Cylinder.
Building own objects: MeshPart
Imported 3D Models: Asset, ModelEntity
Modeling Environment: Terrain, SkyDome, OceanRectangle, Environment, Fog
Cogs.Component is the root class of all Components. Components are normally auto-created when creating an entity. Each component handles part of the entity functionality.
Important Reoccurring Components:
Each Cogs instance runs in a HTMLCanvas. It is normal for an application to do html set up the layout and pass the HTMLCanvas to Cogs.
The application can handle events from the Canvas and draw in html overlays, but should not call OpenGL in the canvas.
The application cannot just delete the canvas to remove the 3D graphics. See below steps required for proper Cogs shutdown to avoid memory leaks.
See Hello world tutorial with Angular and Cogs for details.
Cogs consists of two parts. The graphics engine being a WASM module and the application interface in Typescript. The browser sees the WASM module as one large memory block, starting at 60+MB, growing by complexity of the graphics.
Calling:
control.release();
Shuts down Cogs rendering and removed all internal links to the large WASM module and its owning canvas. The WASM module can only be garbage collected after the canvas is garbage collected. The application must therefore also break all its link to the canvas. We have observed that dangling events listeners stops the canvas from garbage collection.
Never pass a lambda function as event listener: canvas.addEventListener("resize", function () { app.resize(); });
Always pass a function that can be removed as a listener on shutdown.
canvas.addEventListener("resize", this.resize);
...
control.release();
canvas.removeEventListener("resize", this.resize);
Test early by garbage collect browser memory before/after shutting down Cogs.
In TypeScript the data type of the fields are available using intellisense and more info can be found following the definition of the field.
Basic rule is to always set or get field values.
component.field = newValue;
const value = component.field;
Treat fetched values as read-only.
Cogs is written in C++ using either float or double for these types.
In Cogs.js the field types are:
C++ float: vec2, vec3, vec4, quat, mat4; // Imported from npm gl-matrix
C++ double: Cogs.dvec3 // Defines a subset of gl-matrix vec3 using double precision.
C++ double: Cogs.dvec2, Cogs.dvec4; // Aliases using Float64Array.
C++ uint32: Cogs.uvec2, Cogs.uvec3, Cogs.uvec4; // Aliases using Uint32Array.
C++ int32: Cogs.ivec2, Cogs.ivec3, Cogs.ivec4; // Aliases using Int32Array.
The gl-matrix types contains static methods for creating multiplication etc. Do not use Cogs.vec2, Cogs.vec3 etc these definitions are deprecated.
For access to single and double precision 3D vector:
import { dvec3 } from "@kognifai/cogsengine";
import { vec3 } from "gl-matrix";
Understanding precision is important when comparing old vs new values. JavaScript numbers are double precision while Cogs use mainly single precision.
Also note that the gl-matrix vec3 can be either number of Float32Array.
{
const oldPos = vec3.fromValues(1.2345, 2.3456, 3.4567); // Using JavaScript Float32Array.
cube.transformComponent.position = oldPos;
if (cube.transformComponent.position[0] === 1.2345) // Will fail. Gets position as 'float' from Cogs.
if (cube.transformComponent.position[0] === oldPos[0]) // OK oldPos is float.
}
{
const oldPos: vec3 = [1.2345, 2.3456, 3.4567]; // Using JavaScript 'number' array.
cube.transformComponent.position = oldPos;
if (cube.transformComponent.position[0] === 1.2345) // Will fail. Gets position as 'float' from Cogs.
if (cube.transformComponent.position[0] === oldPos[0]) // Will fail oldPos is number, e.g. double.
}
// Safer:
cube.transformComponent.position = vec3.fromValues(1.2345, 2.3456, 3.4567);
const oldPos = cube.transformComponent.position;
...
if (cube.transformComponent.position[0] === oldPos[0]) // OK Fetching both from Cogs.
For array type fields the whole array must be set or get. It is not possible to get the array value and change the array contents.
Cogs.Light light = runtime.Light.create("MyLight");
const camera = runtime.scene.camera;
// OK. Set list of cameras to use for light
light.lightComponent.cameras = [camera];
// ERROR Does not work. Changing set array field is not detected.
light.lightComponent.cameras.add(camera);
Some fields are array of 2D/3D/4D vectors e.g. vec3 and friends. These are represented as Float32Array storing X1,y1,z1, x2,y2,z2 etc. Typed arrays are used for efficiency as the Cogs representation is Float32. See for example Mesh.positions.
Note that the definition of the gl-matrix vector and matrices are either Float32Array or number[]
(example: type vec3 = Float32Array | [number, number, number];).
Same for dvec3: type dvec3 = Float64Array | [number,number,number];;
const camera = runtime.scene.camera;
camera.transformComponent.position = vec3.fromValues(1000, 0, 0);
camera.transformComponent.position = [ 1000, 0, 0 ];
camera.transformComponent.coordinates = dvec3.fromValues(40000, 0, 0);
camera.transformComponent.coordinates = [ 40000, 0, 0 ];
Array field types must use type specified in TypeScript definition.
Note that positions field below is an array of 3D positions.
// Definition:
export class Mesh {
...
set positions(values: Float32Array);
}
const mesh = runtime.resources.createMesh();
// Correct:
mesh.positions = new Float32Array([
1, 0, 0,
1, 1, 0,
0, 0, 0,
]);
// Error: Compile error in TypeScript. Runtime Failure in JavaScript
mesh.positions = [
1, 0, 0,
1, 1, 0,
0, 0, 0,
];
Some fields refer to other entities:
export class DepthAxisComponent extends Component {
trajectory: Entity | null;
The type system always refer to the Entity base class,
the entity type is not so important,
but the usage of the field expects a certain type of component in the entity; here: TrajectoryComponent.
Entity referred to must be created by the client or loaded in a Scene as a root entity. Loaded scene child entities cannot be used.
Reading field value will read the current state from the graphics engine.
const camera = runtime.scene.camera; // Get default camera Entity
// Gets the current camera position
const position = camera.transformComponent.position;
There are some exceptions for some field types where values set in a Scene file will not be read back. Any value set by the client will be returned.
This is normally only a problem when accessing entities in loaded Scene files.
const plane = runtime.MeshPartWithMaterial.create();
plane.meshComponent.meshHandle = myMesh;
// OK: Access the mesh set by client:
const aMesh = plane.meshComponent.meshHandle;
// Access predefine entity created in Cogs C++.
const headLight = runtime.scene.headLight;
// ERROR: Will return undefined value:
const cameras = headlight.lightComponent.cameras;
Resources like 3D models, textures etc are handled in class Resources.
Resources created by the client must be released after use.
const texture = resources.loadTextureViaCogs(imageUrl);
myEntity.materialComponent.diffuseMap = texture;
// Release texture handle, still used in 'myEntity'.
resources.releaseTexture(texture);
// No usage of 'texture'. All resources freed. Either null texture or destroy entity
myEntity.materialComponent.diffuseMap = null;
runtime.destroy(myEntity);
Releasing the resource releases the reference returned when loading the resource; it can still be in use, but any lookup is not possible.
Common pattern often seen in code is:
myEntity.materialComponent.diffuseMap = resources.loadTextureViaCogs(imageUrl);
Set and forget will never release the texture. OK if the entity persists throughout the application lifetime, but not if leafing through images.
TextureResource objects are not purely handler in Cogs.Core. Releasing the resource in Cogs.js should be delayed until the resource is no longer in use.
Other resource objects can be released after setting on fields.
this.myEntity = runtime.Cube.Create();
this.diffuseMapTexture = resources.loadTextureViaCogs(imageUrl);
myEntity.materialComponent.diffuseMap = diffuseMapTexture;
// Later cleanup.
cleanup() {
if (this.myEntity) {
runtime.destroy(this.myEntity);
this.myEntity = undefined;
}
if (this.diffuseMapTexture) {
diffuseMapTexture.release();
this.diffuseMapTexture = undefined;
}
}
Note that all resource loading is a-synchronous. When creating an entity and applying a texture the entity will normally first be displayed without the texture, only later - with the texture when loaded. When changing the texture, there will not be a smooth transition to the new texture.
It is possible to store the texture handle locally and applying the texture when loaded.
// Add callback when creating the Cogs.js Control.
control = await Cogs.Control.create({
...
textureLoadCallback: myTextureLoadCallback,
};
/** Called when texture is ready */
function myTextureLoadCallback(id: number, code: number) : void
{
// Also possibly release old texture if changing image.
if (id === texture.id && code === 0) {
cube.materialComponent.diffuseMap = texture;
texture = null;
}
}
cube = runtime.Cube.create();
// Start loading texture.
// Use ForceUnique to avoid no callback if reusing URL.
texture = resources.loadTextureViaCogs(imageUrl,
{ textureLoadFlags: Cogs.TextureLoadFlags.ForceUnique });
Sample code can be seen in file: tests/src/app/TestImage360.ts
See debugging note below on how to check loaded resources.
Picking functions are found in class Scene. Two types of picking are supported.
Note that all coordinates in below methods return World coordinates. Methods for World Coordinates end with 'World'.
Old versions of Scene methods will return Engine Coordinates (ex: scene.getPickedEntity).
The interface contains several methods, old and new. Recommended methods in class Scene are:
Basic picking:
scene.getPickedEntityWorld
More advanced Pick from position:
scene.getPickInfoWorld(...)
scene.getPickInfoAllWorld(...)
More advanced Pick from ray:
scene.getPickInfoFromRayWorld(...)
scene.getPickInfoAllFromRayWorld(...)
Sample code can be seen in picking example or in test: TestPicking.ts
See Cogs Coordinate Systems for more information.
This coordinate system represents the mesh points prior to any transformation. Expected to be single precision. It is recommended to use coordinates close to the origin or pivot point of the mesh. Any precision lost using large offsets in a local coordinate can never be regained.
When an entity has a parent on its hierarchy the local coordinates of the mesh are always relative to those of the parent and transformations will be concatenated from root to leaf in order to take the mesh points to engine space or world space.
Instead of using large coordinates inside entities, e.g. mesh positions in a GIS application, it is better to use coordinates relative to the entity's transformComponent.coordinate (not using transformComponent.position). For best results this should be the world center of the bounding box for the entity. Note that only non-child entities support this.
World Coordinates - This coordinate system is introduced to handle large offsets from origin where normal 32-bit floating point numbers give low precision and a lot of numerical problems. One example is rendering using UTM Projections.
The absolute world coordinates are always dvec3, e.g. double precision offset is specified using:
import { dvec3 } from "@kognifai/cogsengine";
control.scene.origin = dvec3.fromValues(600000, 7000000, 0);
control.scene.origin = [ 600000, 7000000, 0];
The scene origin can be controlled automatically in a navigation module. Move the origin in steps as changing the origin is an expensive operation. Recommended origin position is the Camera target position. Calculated from camera position, rotation and focusDistance (see class CameraHelper).
myEntity.transformComponent.coordinate = [ 600000, 7000000, 0];
Avoid specifying a large offset in transformComponent.position as this has lower floating point precision.
Calculate World coordinate translation for a root TransformComponent using:
const world = dvec3.add(dvec3.create(), transformComponent.coordinates, transformComponent.position);
The translation sent to the graphics engine is
const delta = (myEntity.transformComponent.coordinate - scene.origin) + myEntity.transformComponent.position;
Keeping these deltas small gives best rendering results.
Also known as relative world coordinates. This coordinate system is used by the render engine with shaders etc. Also used for getting pick results, bounding boxes etc. engine coordinates uses vec3, e.g. single precision. This coordinate system changes when changing the Scene.origin. It is important to keep the origin of the engine coordinates close to the center of the area being rendered to avoid precision problems. For example in UTM coords (offset about one million) precision of 6-7 digits implies that a triangle a few centimeters in size may be degenerate.
Transforming between World and Engine Coordinates:
const engine = dvec3.subtract(dvec3.create(), worldCoordinate, scene.origin);
const world = dvec3.add(dvec3.create(), engineCoordinate, scene.origin);
// Use methods returning World Coordinates. Picking Bounds etc.
Coordinates in 2D window. Note that these coordinates may have to be scaled by DPI scaling.
Cogs uses origin in lower left corner of the Window.
Coordinates in 2D that allow addressing individual pixels in a texture buffer. Usually aliased as (u,v) coordinates, they map to the horizontal and vertical values of the 2 dimensional array representing the texture surface. Texture coordinates are typically in range from [0..1]. Under/overflow is handled according to Material setTextureAddressMode.
control.runtime.setVariable('gui.enabled', true);
The GUI allows inspecting (and modifying) defined entities, resources etc.
If the Cogs.Control 'control' is a global variable, debugging can also be enabled from the Browser Console.
By default there is no support for manipulating the 3D scene. Cogs.js contains a Component supporting basic 3D navigation using mouse / touch. Add this to the default camera using:
const cameraController = runtime.OrbitingCameraController.create();
runtime.scene.camera.components.add(cameraController);
The application can navigate to given position / orientation using fields in the OrbitingCameraController. The system contains utility classes that can help to control this:
Resources:
class Cogs.CameraHelper{}
class Cogs.CameraUtils{}
Example: 0063_OrbitingCameraControllerExample.js // Shows usage.
runtime.scene.viewAll(); // Move camera to put all entities in focus.
The CameraUtils class contains utilities to help focus no a specific bounding-box.
For DIY navigation the package @kognifai/cogsinteraction contains TypeScript classes
that can be used as reference / basis for own development.
Cogs bridge is an internal class for communicating with the Cogs engine. Clients shall not call Cogs bridge. The Bridge functions interfaces the Core Cogs Engine Web Assembly component.
Yes. Using WebXR (not a final standard. W3C Candidate Recommendation Draft, 24 August 2022).
Cogs.js contains example code using WebXR.
Basic rendering using WebGL and WebAssembly are supported by all modern browsers.
WebGL2 allows more advanced rendering and is the recommended target. To target Apple devices WebGL1 target may be needed for performance reasons.
Cogs stores rendering data in its WebAssembly module. There is a limitation in size of total WebAssembly memory of 2GB. Browsers may put stricter limitation on memory usage, for example on IOS devices we have seen limit of ~384MB.
Testing on deployment devices is highly recommended. Test for:
Cogs consists of a Core part plus a number of extensions. The available extensions can be configured at build-time.
The extension must be loaded before any extension Entities or Components can be used.
For a TypeScript user the extension entities/components will seem to be available (ex runtime.PoTreeModel.create()), but the actual create() code is not created until runtime.loadExtensionModule("...") is called. E.g. runtime.Potree is undefined.
if (!runtime.loadExtensionModule("Potree")) {
throw ....;
}
Intellisense will show name of the extension owning Entity/Component.
If the entity is created, the component can be access directly. Picking will return the base class 'Entity' not defining any components. Accessing components can be done using either:
function usingComponents(entity: Cogs.Entity, runtime: Cogs.Runtime) {
const xc1 = entity.getComponent(runtime.TransformComponent); // -> TransformComponent | null
const xc2 = (entity as any).transformComponent; // -> TransformComponent | undefined
const xc3 = (entity as Group).transformComponent; // -> TransformComponent | undefined
}
xc1 is accessed by looking up the given component type using getComponent method. xc2 and xc3 are accessed casting the object type to allow accessing the component as a property of the JavaScript object.
It is possible to create and add components to an entity.
Rules:
const cameraController = runtime.OrbitingCameraController.create();
camera.components.add(cameraController);
cameraController.distance = 50;
// Cleanup
camera.components.remove(cameraController);
Both added and predefined components can be removed using 'components.remove':
// Remove materialComponent to use render component MaterialInstance instead.
const cube = runtime.Cube.create();
cube.components.remove(cube.materialComponent);
A custom entity can be defined in Scene Asset format using JSON section "templates".
Defining custom entities also allows custom initialization of component fields,
as done for Cube entity: "MeshGeneratorComponent": { "shape": "Cube" }
Below a custom MyGroup entity is defined:
const entityDef = `
{
"templates": {
"MyGroup": {
"description": "Custom Group",
"components": [
"TransformComponent",
"SceneComponent",
"CurvedEarthPositionComponent"
]
}
}
}`;
This entity is now available for creating using:
runtime.Entity.loadDefinitions(entityDef);
const myEntity = (runtime as any).MyGroup.Create();
See below how to add TypeScript definition of an entity for Type-safety.
Cogs.js does not expose all Components defined in Cogs. It is possible to add runtime creating of a Component using:
/** For type safety and field access*/
declare class CurvedEarthPositionComponent extends Cogs.Component {
position: vec3;
}
declare class MyGroup extends Cogs.Entity {
get transformComponent(): Cogs.TransformComponent;
get sceneComponent(): Cogs.SceneComponent;
get curvedEarthPositionComponent(): CurvedEarthPositionComponent | undefined;
}
// Register in Runtime class.
runtime.Component.createRuntimeFactory("CurvedEarthPositionComponent");
// Create Custom Entity:
const myGroup = (runtime as any).MyGroup.create("MyGroup") as MyGroup;
// Create component.
const earthComp = (runtime as any).CurvedEarthPositionComponent.Create() as CurvedEarthPositionComponent;
Cogs.js does not expose all Entities defined in Cogs. It is possible to add runtime creating of an entity using:
declare class Overlay extends Cogs.Entity {
get transformComponent() : Cogs.TransformComponent;
get sceneComponent(): Cogs.SceneComponent;
get overlayComponent(): OverlayComponent;
get spriteRenderComponent(): SpriteRenderComponent;
}
runtime.Entity.createRuntimeFactory("Overlay");
const overlayEntity = (runtime as any).Overlay.Create("myOverlay") as Overlay;