Resource Management

A 3D engine manipulates a lot of data, be it geometry, shaders, textures, render targets and so on, and so it must manage them.

The easy solution is to simply let the engine's user handle the memory cost associated to having all those items in the world, but it's neither efficient nor elegant. A better approach is to make the engine ease the work of the user as well as making it more capable, by managing its resources given a memory budget.

The idea is to have the engine only store in VRAM what's required for the given frame, to use the RAM as a cache for the scene around the player, and to use the Disk to store the world. In fact, the system doesn't even require the world to be stored on disk, as it allows procedural generation.

Buffer Management

There are two types of geometry description buffers any 3D engine has to handle: Vertex and Index Buffers. The Vertex Buffer stores vertices, the Index Buffer stores primitives, but both are ultimately the same thing: a simple memory area.

A naïve approach is to simply have a counter that tells how much memory is consumed and allocated or free as required, using a Least Recently Used (LRU) approach. That's not a good solution, as it fragments the memory, and can slow down the engine a fair bit by requiring allocating or freeing memory within a frame.

A way to solve memory fragmentation is to first reserve an area of memory, and then slice it into chunks of memory of various size, keeping everything in an LRU manager. With such a system there's no memory fragmentation or slow down due to allocating or freeing memory, however there's a big potential to waste lots of memory. Too many big chunks compared to what’s required and the engine wastes memory and has to reload chunks often, leading to slow performance; Too many small chunks, and the engine can’t find room and need to merge chunks within the frame, leading to slow performance. There simply is no reliable way to know how to slice the original memory area so that there is as little waste as possible.

A thread on forum described a whole caching system called the Sliding Slot Cache. I wasn't interested in all the details of that particular implementation, but I liked the concept of an adaptive cache, using statistics to self modify according to current load. And so I kept the idea and wrote my own implementation of it: instead of having big areas split into chunks as in the original implementation, I chose to have chunks being autonomous. For example, a chunk is a Vertex Buffer, not an area of a Vertex Buffer. This was because OpenGL 1.x/2.x could only discard a whole buffer but not a subset of a buffer, unlike Direct3D 9 (D3D9).

With OpenGL 3.0 offering that capability, I expect to implement something closer to the system described in the thread, having Vertex Buffers of the same size being subdivided into chunks.