Legends of Aereven: Lunar Wake is a 2D Zelda-like I made in 2020. As I continue my gamedev journey with other projects, I wanted to take the time to look back on it. I'm planning a series of posts acting as a postmortem for the project.

This first part is dedicated to the inner workings of the custom engine built for Lunar Wake and my next projects. The next part will be about the tools and pipeline, and then we will talk about design and production.

Why C#?

As a kid, I learned programming so I could make games. I started with Basic-inspired languages (first the game-oriented DarkBasic, then the general purpose PureBasic) before switching to C++ in highschool. In many ways, the inherent complexity of the language was a distraction: while I learned a lot about programming in general (and a lot of bullshit only applicable to C++), I wasn’t making a lot of progress on the gamedev front.

After a few years of making prototypes with C++ and Irrlicht, I had to use C# for a school project (because csc.exe was the only compiler already installed on the school computers, as part of .NET)... and I fell in love. C# was everything C++ wasn’t: simple, clean, elegant (and it was before C++11, which added another layer of complexity and legacy considerations to the pile). 

I played with an Irrlicht .NET binding for a few months, but then came Microsoft’s XNA, and it was another revelation. XNA was lower-level than Irrlicht, yet not as low as SharpDX. But more importantly, like C# itself, it was clean and elegant. The API made sense. With C# and XNA, programming games was all pleasure and no pain.

There was no looking back.

GameLib and SPF

Flash forward a decade or so, and XNA is dead, because Microsoft seems to get a kick from killing their best products and initiatives.

The rising star at the time was Unity, but I hated it from the start. I hated the philosophy (framework vs toolkit) as I felt constrained by the engine. And I hated the execution: back in 2010, the Unity Editor would frequently crash, and I would lose work. Once my trust was lost, there was no recovering it. Add to that the poor Visual Studio integration and the old Mono runtime, the non-idiomatic use of C# and the lackluster support for 2D... Coming from XNA, the Unity experience was miserable.

So, what were my alternatives? I tried MonoGame and FNA, both open source reimplementations of XNA. They were fine. A few bugs here and there, and the shader pipeline of MonoGame was a bit cumbersome, but it was workable. I used FNA for a couple of years. 

Around that time, Sony took a page from Microsoft and launched the Playstation Mobile initiative, which meant I could use a fork of MonoGame on the PS Vita. And I very much wanted to! But while XNA worked with the same types and namespaces across PC, XBox 360 and Windows Phone, here I had to use different (yet 99% identical) types for the PC and Vita targets.

That’s how GameLib came to be: I extracted the math classes from MonoGame in a separate assembly, so my gameplay code would be independent of the actual XNA-fork I would use for the rendering. And then I started adding stuff, from general utilities functions to very specific gameplay behavior. After five years it snowballed to the point where I had a full Tactical RPG engine hidden somewhere in GameLib… Oops?

But Sony took another page from Microsoft and killed Playstation Mobile before I could do anything with it. By then I was a bit tired of FNA, which was intended to be a 1:1 reimplementation of XNA for archive purposes and would never receive new features.

I don’t need something so general, I thought. So I started p/invoking Win32 and OpenGL directly to write a “platform” library. All this interop quickly became a pain, so I turned around and wrote the whole thing in C++ and only exposed my own API to C#. Also, I ditched Win32 to build on top of SDL2  and SDL_mixer. The platform handled window management, input handling, rendering (with a built-in sprite batch), audio and I even added Bullet for some simple physics stuff. I called it “SPF” for “Simple PlatForm”, but two shipped games and a dozen prototypes later, I’m not sure the “Simple” adjective fits anymore. I intended it for 2D pixel-art projects and now I’m using it to make 3D stuff with multiple render passes, custom shaders and stencil buffer abuse.

I still actively maintain SPF, which is open source: https://github.com/Dreamnoid/SPF/

Dream

The problem with GameLib was that it provided reusable blocks, but you had to glue them together.

For every new project, I had to write a lot of boring boilerplate code just to get to something playable. Sure, the controller input and collision routines were provided by SPF and GameLib, but how many times do I have to write the same damned velocity or gravity code to bridge the two? Map loading and autotiling are in GameLib, as well as a Camera2D class, but how many times must I write the same tilemap rendering code with scrolling considerations? That’s not a new problem, that’s not fun.

I grant you, it’s only an issue because I have no discipline and I often start new projects on a whim. But it’s easier to change your tech than your nature. So that’s what I did.

Enter Dream. When it started, Dream was 99% GameLib with the namespace changed. But the goal of Dream was to provide more than reusable classes, but reusable glue too.

Yet, it had to be optional, otherwise I would end up with another Unity. I would have built a prison tailored for me, but it would still be a prison.

Dream is architectured in modules, and only one is mandatory: Dream.Core, which is the direct descendant of GameLib. It defines a lot of utility classes and functions (math, collisions, serialization, generic data structures, etc) that can be used in a standalone fashion, but also interfaces necessary to describe what I call a "Platform" (IWindow, IInputProvider, ITexture, IRenderer, IFilesystem, etc).

Then I have a Dream.SPF module that acts as a "Platform" implementation using SPF. The Platform interface was described with SPF in mind, so it's almost a 1:1 bridge between SPF and Dream. Those are the two modules I basically use every time.

Then you have two other kinds of modules: those dedicated to certain kind of games or gameplay features (eg: Dream.Vino provides staging and dialog features for Visual Novel or RPG games) and those used for tools and pipelines (eg: Dream.Editor is an IMGUI-type tool-builder and Dream.Drawing includes classes to manipulate bitmap or raytrace sprites from 3D models, etc.)

There are currently 13 modules in Dream!

ECS

So, reusable blocks is easy and all, but how do you reuse the glue between them?

I was first made aware of the Entity/Component (EC) and Entity/Component/System (ECS) concepts through the “Evolve your Hierarchy” and T-Machine blogposts, and it was eyes-opening. Around the same time, the notion of Data Oriented Programming was also gaining traction, with the Sony paper blasting OOP or Mike Acton’s post-its. Problem is: the two ended up confused in my mind (and to be fair, in a lot of literature too).

The full ECS system was always described to me as a performance thing. Entity are just IDs! Components are stored sequentially! Cache friendly! Except I couldn’t wrap my head around it. Wouldn’t you need more than one component to do anything useful? Wouldn’t that cause random accesses? Surely I was missing something. (As it turns out, yes, I missed the fact that in that kind of design, data is constantly shuffled around in memory). But the games I was making could run on a GBA, so what did I care for cache misses?

In 2017, the Overwatch team gave a GDC talk about their ECS implementation, and in 2019 it was finally made available outside of the Vault. It was everything I hoped it would be (I had tracked a Chinese transcript before, but Google Translation butchered it). They didn’t talk about performance. They said ECS helped with side effects. ECS helped with order of updates. ECS helped with coupling. ECS made it easier to organize gameplay code. Finally an ECS talk about my concerns!

And with that, another side became apparent: ECS would help reusability. Components don’t need to know about each other, as they’re entirely piloted by higher-level systems. It’s a bit like Duck-typing, where the mere presence of properties or methods on an object is enough to make it eligible for some behavior, without the need for a rigid agreed-upon interface.

The ECS would be my glue.

So, how does that work in Dream?

Entities are real. Not just nebulous IDs, sorry data-oriented purists. They come with a few flags (like Terminated), a position (a Vector3 that can be used in both 2D and 3D contexts) and a list of components. (Only one of each type is allowed.)

Components are barely more interesting. They only store data. I do allow ease of use methods, like setters that will change multiple properties at once in a coherent manner, but that’s it. They also have an Enabled flag. If disabled, it’s like the component never existed. It won’t be queried by the systems. So, let’s say I have a ghost enemy that periodically disappears while still moving around: I just need to enable/disable the Sprite component to make it eligible or not for rendering.

Scenes are basically a collection of entities and registered systems. You can queue spawn operations at any time, but entities are only created at the beginning of a frame, and only removed at the end. That way I don’t have to worry about half-updated entities.

And then we have systems. Systems do not hold state (though they may hold configuration settings or resources). Instead, they query the scene every frame for tuples of components. Like ‘give me all entities with both a Physics and a Sprite component’, and then the components are accessed from the entity. That system may for example change the sprite to reflect the current physics state. In practice, with the position stored directly in the entity, I never had to query more than 4 components at once.

(Now is a good time to tell you I don’t do multithreading. Because I don’t need to. But if I did, I would need to be a bit more strict about separating reading and writing operations, so I can run the systems in parallel.)

Each frame, the scene will run every system in the order they were registered. So what happens when stuff need to be updated in a specific order? You could try to register the systems carefully, in order, but it’s a losing battle. Instead, the ISystem interface provides a few callbacks that act as ‘passes’:

• Gather is called first to get info on the current state of the game. Like the list of actors that will contribute collisions. Or syncing the player’s entity health component with the HP written in the save file.
• Controller is a chance to decide how entities will move before the physics pass. It’s here that I translate the user inputs into a velocity vector. It’s also here that enemies AI decides what to do.
• Physics will try to resolve all movements and collisions. It will handle gravity, if enabled, check if an entity is grounded, etc. The beauty of ECS is that you move all entities at once, so you avoid the bugs that usually come with some entities being fully moved while others have yet to start (moving platforms are straightforward with ECS).
• Interact is another chance to decide what entities will do, but once physics is resolved. Typically, I will use this pass to handle interactions (like a ‘Talk’ prompt), triggers or hitboxes.
• Lifespan is near the end of the frame, opposite to Gather. It’s where I decide which entities should be terminated or continue to live (like removing a played out FX).
• Animate comes at the end to update the graphics so they can reflect all that happened in the frame.
• Render is used by RenderSystems like Dream.Pixel and Dream.Polygon, but also by the UI RenderSystem.

The Gather method will be called for each systems, then the Controller method, then Physics, etc. So order-dependent code is always in different passes.

One last thing: scenes have components too. They’re usually a different set of components than those used on entities. More like the ‘singleton components’ of Overwatch. Those components are directly accessed from the scene, always in a GetOrCreate way. Think of it as a way to exchange data between systems without them having to know about each other. For example, the CameraSystem will write to a Camera2D singleton component that will later be read by the RenderSystem to render the correct viewport. If I do not register the CameraSystem, the RenderSystem will get a Camera2D component with default values, so everything continues to work fine. I can replace the CameraSystem with a different one, or add other systems between the two to change those data (ScreenshakeSystem!).

Hopefully you’re starting to see how it can help modularity ;)

Okay, the actual last thing: System bundles. I’ve found that some systems are ubiquitous. Everything dealing with the lifetime of entities, or their hierarchy, or triggers, etc. Every games need them. So I bundle them together to register them all at once. The CoreSystem is the most obvious SystemBundle, and I always register it. But you can also have an RPGSystem that will bundle a collection of systems that work well for making an RPG, for instance.

Subroutines

ECS is great for organizing what happens in a frame, but games often have to coordinates stuff happening over multiple frames. For that I use subroutines, made easy in C# thanks to the yield keyword.

I often use subroutines as coroutines. Mostly for entity behavior. If you remember the example about the ghost becoming invisible, that behavior can be driven by a coroutine that will enable/disable the components after some time has passed, or if the player is near, or randomly.

So when is a behavior a component/system and when is it a coroutine? My rule of thumb is that if the behavior is reusable, it’s a component/system. If it’s unique, it’s a coroutine. A HorizontalPatrolComponent can be used for both Goombas and Koopas, but a Bowser boss fight would be a unique coroutine.

Like I said, the whole ECS thing is an option. Scenes are game states, but game states need not be scenes. Most of them aren’t, and are instead driven by subroutines. The most notable examples are menus.

Some states are both. For example, if I were to make a 3D turn-based JRPG battle, I would have a scene with actors for every fighter, rendered and animated with their respective systems, but the logic would be driven by a subroutine. Turn-based systems are usually a better fit for subroutines than ECS.

Pool allocator

C# is a garbage collected language, which is great but also terrible. Great because when you don’t care about memory, you don’t have to. But terrible because when you do care about memory, there’s not much you can do. 

Thankfully, you usually don’t need to do much.

The bane of every C# game is when the GC decides it’s a good time to clean up house, right in the middle of a frame. .NET’s GC is fast, but not 60 FPS fast. So how do we make sure it doesn’t run during gameplay? By not allocating memory.

That’s why the Entity class and all components implement an IRecyclable interface that moves their initialization from their constructor to an explicit Recycle() method. New instances are requested through a global pool allocator, that will only call ‘new’ if no instance is ready to be reused. When the code is done playing with the object (e.g. an entity has despawned), it is sent back to the pool to be recycled. That simple system removes most of the allocations happening during gameplay. (What’s left is due to .NET being allocation-happy, and usually remedied by reusing large-capacity lists, caching delegates or avoiding AddRange and Sort.)

(And yes, I do allocate memory… up until I don’t need to anymore. .NET allocates memory in pages, so a single new is pretty fast. My memory consumption curve starts linear but quickly ends up flat. And I manually run the GC during scene transitions, while the screen is black, just in case.)

Prefabs

Creating an entity is as simple as calling scene.Spawn() and adding components to the newly recycled entity.

Prefabs rely on that. They’re semi data-driven: built in code, as functions that can receive extra information. The prefabs can use that extra info to adapt using any code imaginable.

It can be really simple, like the Chest prefab offering a different item depending on the input. Or a door requiring a different key. But it can get more complex, like the NPC prefab that will completely change in appearance and behavior according to the NPC definition.

I can also run logic to make the spawning conditional. It was super helpful for Lunar Wake because the main source of bugs after release was props respawning at the same position as the player, making them stuck:

It’s a simple yet powerful system.

Since then, I tried making it fully-data driven. It simply wasn't worth it: I ended up complicating things by orders of magnitude and it required extensive tooling. Turns out, for a lone programmer, writing game-specific code is okay! That being said my current approach is still a lot more data-driven than what I had for Lunar Wake. The code only defines reusable broad behaviors and everything else is configured in data.

Anyway, the reason I can do data-driven stuff is thanks to the...

Serialization

Serialization is nothing new in .NET, but I don’t use the builtin system. You may wonder why. Sometimes, I do too.

After The Lightkeeper, I briefly prototyped a 2D JRPG. I wanted to use Lua for the scripting, but the .NET bindings were a pain in the ass. And I didn’t need the full power of Lua anyway. And you may have gathered by now that I like reinventing the wheel (and why wouldn’t I, it’s fun!). Even worse: I love writing programming languages. So I wrote RPGScript. RPGScript is a simple, lightweight, 100% managed Lua replacement. (It’s also open source: https://github.com/Dreamnoid/RPGScript)

For a few projects, I used RPGScript for both data definition and scripting. But I quickly (and painfully) realized that mixing code and data together could be problematic. At first you’re very reasonable, only using code to describe behaviors like what happen when you talk to an NPC. But then you start abusing that power to develop a macro system and everything goes to shit.

So for Lunar Wake I started from RPGScript but removed functions. And then I changed the syntax to look exactly like JSON.

The way I do automatic (de)serialization is by having an IData interface exposing a Visit(IVisitor) method. This method will describe the properties of the object (usually name+type, but it can be a bit more involved if need be).

Two examples of Visitors are the serializer and deserializer, but the Dream.Editor module can also leverage that to create automated UI.

Beyond that extra flexibility, my solution has the advantage of not relying on reflection, which means it should be a bit faster (it’s certainly a lot easier to use). Also, you don’t need to reference NewtonSoft packages.

Scripting

Now that I have replaced RPGScript with JSON, what of the scripting?

All languages are trees before being compiled, but in RPGScript the AST was executed directly, without being converted to opcodes. After migrating to JSON, I quickly realized I could write my code using only lists. It works, but the syntax is unwieldy, especially when you’re using an AZERTY keyboard.

Writing code with nested lists? If only I knew a language that worked that way… Brackets are hard to type, so maybe if I switched to parentheses… Removing unnecessary double quotes… Hmm... That DOES sound familiar.

Yeah, it’s freakin’ Lisp!

My Lisp parser actually creates JSON lists under the hood, so the virtual machine didn’t even have to change, but the syntax became a lot nicer.

So... did it work?

It did!

It really did.

As we will see in the next parts of this postmortem, this project went through a few false starts before it became a Zelda-like. So by the time I started working on Lunar Wake proper, Dream was pretty much up to speed. Map rendering, collisions handling, hitboxes interaction: all that was already working.

The most code I had to write at first was the PlayerControllerSystem that handled the particular Zelda-like movement, sword swing and other tools use. It got gradually more complex when I implemented advanced gameplay systems like falling in pits, swimming, skidding on ice, crossing chasms with the hookshot, etc. But none of those features took more than a day or two to implement.

Likewise, implementing enemies was a breeze. I could go from a rough idea to a working prototype in very little time. And the ECS even suggested new gameplay ideas! When you have a bunch of systems ready to go, you're tempted to mash them together to see what happens. That's how the Ice Rod in the Ancient Forge extension came to be: I could already spawn platforms, make them temporary and turn their surface slippery, so the idea came naturally.

So yeah, big win for ECS and reusability. All of my current prototypes and more serious projects use Dream, and they reuse a lot of code. I regularly promote code written for a specific game to Dream.Core or a dedicated module.

Next time I will show you the tools and pipeline I made and used to create Lunar Wake. See you then!