Bonsai: A Voxel Engine, from scratch

I've been thinking of doing a series of blogs on the journey but .. it's been a journey, which is a lot to write about in full. In short, a few places where I've been able to prefer simplicity:

1. Allocators are all pretty much as simple as you can get. Most memory in the program is bump/arena allocated. There is a buddy-style heap allocator for things that are annoying to arena allocate (strings that can be edited, for example). I make heavy use of temp memory and freelists.

2. Containers are all very straight-forward, and it's definitely a feature. The example I always give here is std::map from C++. On paper, it looks great; it has very good looking properties. In practice, the implementation is a nightmare; it's slow, and has a comically large rebalancing-shaped performance cliff. My containers strive to be simple, with reasonable average and worst-case performance.

3. I wrote my own metaprogramming language instead of using C++ template metaprogramming. Writing an entire programming language sounds like the antithesis of simplicity, but in reality, having a good metaprogramming layer makes your life immeasurably easier in the long run. With strong metaprogramming capabilities, stuff like realtime debug UI and state serialization becomes nearly trivial. Once you start doing versioned data serialization in C++, you quickly realize you need a better compiler (see: protobuf, cap'n proto)

It's actually more efficient to do a hybrid approach, especially at high view distances. Rasterizing triangles is extremely fast, and is basically a perfect primary-ray intersection. Ethan Gore recently did some experiments with raytracing and said that for large scene volumes (his engine comfortably renders the entire 32-bit range, or 4B^3) it turns out to be faster to do raster for primary rays and raytrace shadows/GI.

Still, games like "C&C: Red Alert" used voxels, but with a normal mapping that resulted in a much less blocky appearance. Are normal maps also a performance bottleneck?

> I’ll have to try some of my assets out in this engine.

The asset loading is currently broken in Bonsai after a big rewrite, but it's the next thing on my list to fix! Would love to see your art :)

Thank you!

> It's like looking into the future, will there be a point where you can just render the entire scene in voxels at high enough res where it's indistinguishable from modern 3d graphics?

There are two really big problems getting in the way of this.

1. Data size. One of the ways that my engine gets away with rendering such large volumes is by discarding most of the voxel data after it gets generated and converted to triangles. In the context of a traditional 3D pipeline, where artists hand-author content in zbrush, maya, blender, etc, which is then imported and drawn in the engine .. it gets complicated.

2. Animation. Animating voxel data is hard. I'm going to completely gloss over all the details, but current techniques mostly boil down to playing back 3D video, which is problem (1) on steroids.

That all said, many voxel-based techniques are used in commercial 3D game engines today. Unreal Engine just landed a feature that voxelizes foliage in the distance, and just draws the ~1px voxels instead of textured quads. Voxel Cone Tracing is another prominent example. Fluid sims and many volumetric effects are also commonly done using voxel grids.