What I've been building: a game engine, an agent board, and a voice in my PC

It’s been quiet on here for a while, but not because nothing’s been happening. Since the end of August I’ve started more projects than I’d like to admit, and a surprising number of them actually work now. This post is a tour of all of it: what each thing is, why it exists, and where it’s going.

The short version: I wrote a game engine, then a kanban board for AI agents to work through my tickets on, then a voice assistant that talks to that board, then a tool to post videos of all of it. Along the way I made a few terminal programs because I kept wanting them.

espresso, the game engine#

espresso is my own game engine, written in Rust. It’s the biggest thing here by a mile (well over 400 commits in about three weeks) and the one I’m proudest of.

What’s in it:

• A wgpu renderer: forward rendering, sky, fog, grass and water that move in the wind, particles, physical light units, post-processing, custom shaders and even a path tracer
• Physics through Rapier, wired into the node tree, with a character controller, cloth, joints and ragdolls
• Skeletal animation from glTF and FBX, with IK
• A retained-mode UI: layout, text, widgets, focus, gamepad support, nine-slices, and interfaces that live in the 3D world
• Spatial sound
• Multiplayer over UDP on desktop, and WebSockets and peer-to-peer WebRTC in the browser, so a desktop player and a browser player can be in the same game
• Scene files and an editor that a game embeds, with hot reload, so you can edit a game while it’s running

It also builds to WebAssembly and WebGPU, which is how the demos end up on this site. Each one is exported from the engine and uploaded through the admin area here, and the Lab serves them.

Some favourites:

• sandbox: a physics sandbox in the spirit of Garry’s Mod. Spawn props, ragdolls and cloth, carry them around with a physics gun, weld and tie and paint them with a tool gun, and rocket jump. You can host or join from a desktop, join from a browser, or meet up in a room with a code.
• destruction: a street of buildings that break exactly where you hit them. Each building is one mesh, fractured at the moment of impact.
• holdout: a small twin-stick shooter. Rounds of critters, guns, barricades and turrets.
• magic: a stone circle at night. Hold click to gather a spell, let go to throw it.
• zoo: a hall with an exhibit for every engine feature, each with a plaque explaining it. The closest thing to a guided tour of the engine.
• character, first-game, title-screen, composition and the gpu-test stress test round out the rest.

The engine was called fold for a while (hence the repo name), and it’s espresso now. It’s private for the moment, but the plan is to keep putting demos up as they come.

ultrakanban, a kanban board for people and AI agents#

Here’s the thing about building an engine that fast: I didn’t type most of it. A lot of the work was done by Claude Code agents, and I needed a way to hand them work, keep an eye on it, and review it without babysitting a terminal all day. So I built ultrakanban.

On the surface it’s an ordinary, fast, keyboard-friendly kanban board: boards, columns, drag and drop, tags, priorities, checklists, comments, a full activity log, live updates. Underneath, it’s built for agents to work from:

1. An agent claims a ticket. Every write is atomic, so several agents can work one board and never grab the same ticket.
2. It posts a plan with a size estimate (S, M or L). Per board, I can make anything above a certain size wait for me to approve the plan before it burns tokens.
3. It does the work, checks off the ticket’s checklist as it goes, attaches screenshots of anything visual, opens a pull request and moves the ticket to review.
4. I review. Comments on the ticket or the PR wake the agent back up to address them.
5. The ticket only reaches Done when the pull request is merged, and it gets there on its own.

There’s a lot around that loop: sub-tickets for work that’s too big for one PR, a “merge all” button that merges everything in review in an order that avoids conflicts, a button that asks the agents to fix their own merge conflicts, optional auto-merge once CI passes, notifications when something actually needs me, and an overview dashboard showing which agent is working on what, how long it’s taken and how many tokens it’s used, broken down by model.

Each board’s agent runs as a little loop outside of Claude Code that starts a fresh, short-lived session per ticket, optionally several in parallel in their own git worktrees. I’ll file a handful of tickets before bed and wake up to pull requests.

Jarvis, a voice assistant for my PC#

Once the board existed, I wanted to talk to it. Jarvis is an always-on voice agent for my machine: say “hey Jarvis” and what you want, and it does it and talks back.

It can file and read tickets on ultrakanban (“what’s in review?”), look things up, drive OBS and content-machine (below), run allowlisted shell commands, and hand anything bigger off to Claude Code.

A few things I like about how it’s built:

• It costs basically nothing to run. The wake word, speech recognition and the voice all run locally on open models (on the GPU with CUDA if there is one). The thinking goes through Claude Code on my existing subscription instead of the paid API.
• It’s careful. Every tool has a risk tier. Risky things need a spoken confirmation, and everything goes into an audit log.
• It answers fast. Replies are spoken sentence by sentence as soon as the first one is ready, and you can talk over it or ask a follow-up without the wake word.
• It maintains itself. “Any updates?” tells you what’s new, and “update yourself” pulls, rebuilds and restarts it. It can change most of its own settings too: “talk a bit faster”, “make the wake word less sensitive”.

It’s a Rust daemon (jarvisd) plus a little jarvis CLI, with push-to-talk on a keyboard shortcut and an icon in the top bar. It’s the newest project here and moving the fastest.

content-machine, record once, post everywhere#

I want to post clips of all this stuff, and I don’t want to spend an evening re-cutting the same recording for five platforms. content-machine (cm) is the fix: record once, describe the post in one file, and it renders every cut in the shape each platform wants, with burned-in captions, and releases them all together.

Instagram, Reddit, TikTok and YouTube are first, with this blog in the queue too. Since a lot of what I record is 16:9 gameplay or editor footage, it has a few ways of fitting that into a vertical video: crop around a focus point, or fit the whole thing over a blurred copy of itself. It’s Python on top of ffmpeg, and Jarvis can drive it.

The terminal stuff#

These came first, back in August, mostly because I wanted them for myself. All four are open source.

• textfold: a terminal text editor with normal keys. Ctrl-S saves, Ctrl-Z undoes, Ctrl-Q quits, and there are no modes. Syntax highlighting and language servers are built in, so open a Rust file and rust-analyzer just starts. No plugins, no config file.
• sshman: a two-pane SSH file manager. Your machine on the left, the server on the right, one key to copy either way. Split any pane into a real shell, open several servers in tabs, and treat Docker and Podman containers like hosts. One binary, nothing to install on the server.
• libterm3d: a software 3D renderer for the terminal, with per-pixel lighting and a z-buffer, drawn as two stacked half-blocks per character cell. It comes with a first-person demo you can walk around in.

And this site#

This site got a full rewrite too: statically prerendered Next.js over Firestore, a custom MDX pipeline that compiles posts on the server so articles ship no JavaScript, and the Lab for the engine demos. I wrote more about how it fits together in cool facts about this website.

What’s next#

Expect more demos in the Lab, Jarvis getting smarter, and hopefully some actual video posts once content-machine is publishing for real.

If you want to hear when new stuff lands, the newsletter is the best way.