Little Polygon Game Dev Blog

🏗️ Scaffold Level Editor

Mon, 07 Apr 2025 00:00:00 +0000

On the latest Nightshift Galaxy weekly dev stream I demonstrated the specialized level editting tool I’m building inside the Unreal Level Editor that I’m calling Scaffold.

I gave an impromptu introduction to my motivations and inspirations live, but on reflection the topic deserves a written deep-dive.

In designing tools I’m guided by three high-level goals:

Productivity. As a solo-developer, I need to automate as much of my workflow as possible, to ensure that I focus on places where my specific skill-set adds value, and avoid labor-intensive tasks that aren’t core to gameplay.
Individuality. A risk of using a generic game engine, like Unreal or Unity, and common marketplace assets is producing a generic game. By developing unique systems, I can enable unique gameplay that has market differentiation.
Performance. As an action game with a high skill-ceiling there is no “correctness vs. performance” tradeoff; 60+ FPS is a correctness requirement. Furthermore, in my experience, to confidently reach a larger audience with midrange commodity devices, one can’t simply chip-away at perf problems in the profiler later – one must architect for it.

Scaffold addresses these goals by exposing (i) an interactive interior design tool that prioritizes power-user hotkeys and hackability, (ii) gameplay systems that supplement Unreal’s built-in collision and navigation with opinionated parallel systems, and (iii) data-structures (inspired by 90s game engines) that are efficient by-construction.

Background: Convex Decomposition

To explain the details of the system, we need to first establish some background in Convex Geometry that’s relevant to the design. In a nutshell, the convex hull property describes any shape where the straight-line connecting any two points within the shape is also contained in the shape.

Sample Convex Shapes in 2D

This is in contrast to concave shapes that have notches or holes cut out of them.

Sample Concave Shapes in 2D

Most spatial algorithms in games, particularly raytracing and pathfinding, implicitly rely on the convex-hull property by partitioning complex scenes into a network of connected convex-volumes through a process called Convex Decomposition.

In this data-structure, the faces of the volumes fall into two categories: Walls that close the outside of the composed shape, and Windows in-between shared internal edges that are logically open.

Convex Decomposition

To trace a ray through the scene, we start by identifying which volume contains the origin of the ray (ususally from context). Because of the convex-hull property we know we can extend a straight-line all the way to a point on the edge, which we identify through ray-plane intersection. Then we check if the edge is a “wall,” in which case we’ve successful identified the hit, or a “window,” in which case we go back and repeat the process for the newly-entered volume.

Raycasting within a Convex Decomposition

This algorithm is local and for volumes with a fixed-capacity for the number of sides has O(1) complexity, relative to the size of the scene, as opposed to a brute-force algorithm which inspects every single wall and therefore has O(n) complexity.

We extend this algorithm to loose-props and moving-actors by associating with each volume a list of primitives, what I call an “object-bucket,” which is searched for a hit before advancing to the far-edge.

Raycasting with Loose Object “Buckets”

By assigning a fixed-capacity to buckets, this keeps the cost O(1). When we consider that every actor is doing their own traces to move every frame, that gives us something like O(n) total performance complexity, vs. brute force in which every actor is checking every other actor for O(n²) total performance complexity, which is unacceptable for interactive applications.

If the buckets have variable-capacity, we can avoid the “everyone crowds in one volume” edge-case by recursively subdividing buckets spatially and making a tree-like structure. For a uniform-distribution of actors, this has O(log n) trace complexity and O(n × log n) total complexity, which is the typical threshold in game programming for “fixable via perf profiling.”

Another use-case for convex decomposition is pathfinding. By drawing waypoints in the middle of each “window” we know we can connect them across their shared volumes with straight-lines (again, due to the convex hull property). Furthermore, any actor inside a convex-volume can make their way to any of that volume’s waypoints, and as long as a connected path exists we can draw draw a path between any two points that is free of collision.

Pathfinding on a Convex “Navigation Mesh”

The algorithm to determine the if the path exists is beyond the scope of this writeup, but it’s called the A* Search Algorithm and there’s lots of easily-searchable literature about it online.

A wrinkle in understanding how these concepts apply to modern general-purpose game engines, like Unity and Unreal, arises the architecture of their level editors. Both eskew the explicit construction of a spatial decomposition, and instead expose to the designer a simple list of loose objects that are hand-places one-by-one (“Actors” in Unreal, GameObjects" in Unity). Therefore, to apply these algorithms, dynamic data structures are built at run-time, on the game device, based on heuristic, fine-tuned methods.

For collision and raycasting, the spatial partition is built via an invisible lattice of implicitly-linked convex grid-cells (think like the lattice in Minecraft, except each cell has a object-bucket rather than a solid voxel).

Here’s a 2D “Quadtree” illustrated. The convex decomposition is limited to boxes and object-buckets, but no true “walls”.

The origin of the ray is determined via quantization, and proceeds in the same way outlined above, except the grid-cells have only “window” edges, so we’re only collision-checking the buckets. The step of quickly filtering via local buckets is described in the literature as “Broad-Phase Collision”.

Tracing a ray through a broadphase spatial partition (greatly simplified).

Because a rectangular grid cell cleanly divides into 8 sub-cells, this lattice data structure is called an “Octree.” Due to its simplicity, Octrees are the most common spatial partitions, but other interesting schemes exist, and the solution in production often has several interesting optimizations that are beyond the scope of this summary – but the principle is the same. Both Unreal and Unity used to rely on the PhysX library for this task, though Unreal has transitioned to an in-house solution called Chaos. Another notable solutions are Jolt, which was originally developed for the game Horizon Zero Dawn, and Havok, which has been around a long time, most notably/recently powering Nintendo’s Breath of the Wild.

“I’ll just develop my indie game the same way a multimillion dollar company develops theirs” is not exactly a winning strategy.

For navigation, general-purpose engines rely on hand-placed volumes to perform many continuous raycasts on background threads at runtime to asynchronously populate connected “ground shapes” on which pathfinding can be performed. This process is more art than science, and relies on a vast number of fine-tuning parameters that demand continuous designer tweaking for good results. Unreal, in particular, relies on the Recast library for this task.

Screenshot of the Recast Navigation system, from their official Github page.

The downside of these general-purpose dynamic solutions is that the CPU and RAM requirements are higher than precomputed explicit decompositions, and they’re more likely to exhibit the variable O(n × log n) complexity that demands laborious perf-tuning. Furthermore, they transfer all the work of constructing the patritions to the end-user’s gaming device, rather than allowing static scene data to be “cooked” ahead of time on developer workstations.

Looking Forward by Looking Backward

They’re workable in a large-team production setting, but in my case they resisted all three of my high-level goals. Therefore, I began searching for alternatives – not to replace these features whole-cloth, but the supplement them with parallel systems for specific cases where they’re not well-suited.

The fact that in principle they all share a common data structure gave me this feeling that I should be able to make a tool which does “double duty”.

In searching for inspiration I began thinking about mid-90s shooters, which accomplished a lot of what I want on very modest low-power commodity PCs, in particular DOOM (1993) and DESCENT (1995). How did those games work? And could I exploit those legacy techniques to surpass the performance and productivity limitations of the general-purpose solutions?

DOOM: Binary Space Partitioning

Playthrough of DOOM, developed by id Software in 1993

Loose objects like demons and bullets are pretty sparse in the original DOOM, with wall-collision accounting for most of the runtime complexity. The DOOM level editor allows for mostly arbitrary placement of walls, but then runs a special decomposition algorithm to break it down into explicit convex-regions called Binary Space Partitioning (BSP).

Screenshot of the origin DOOM level editor.

The idea is to pick a wall at random, and consider the subdivide the remaining walls into those that are in-front or behind that wall’s plane (subdiving walls that straddle the plane into two parts). For those two subsets you rinse-and repeat until you’ve exhausted all the walls, and what remains at the leaves of the subdivision are a convex partition of the scene, which can be computed and stored ahead-of-time with level-data.

Diagram from Valve’s Source Engine documentation demonstrating convex-decomposition via Binary Space Partitioning.

The original DOOM simplified these BSPs by imposing the restriction that all walls are perfectly vertical, so the partitioning could be mostly 2D, though later versions of Quake and Unreal Tournament used the same technique with sloped geometry, too.

While it’s interesting to study to understand the history of how broad-phase collision developed historically, I didn’t really feel like there was much for me to exploit

DESCENT: Convex By-Construction

Another interesting case I looked at was Descent, which billed itself as the first “full 3D” shooter with complex volume-filling geometry, as opposed to being mostly-flattened onto the horizontal plane like DOOM.

Playthrough of Descent, developed by Interplay in 1995

I better appreciated for how they implemented this by playing around with the mod community’s level editor DLE, where levels are built-up from convex “segments” that are each six-sided cuboid hexahedrons. At first glance this seems like a restrictive limitation which would give rise to rectangular chambers, but creative linking across edges and corners can actually give rise to pretty much any shape you want with enough creativity. Additionally, restricting faces to all be quads simplifies how textures and materials are applied, as opposed to having to account for weird triangular topology.

Screenshot of the Descent Level Editor. It looks like a mesh editor at first glance, but you’re actually editting linked cuboid “segments.”

In 2018 the original original devs reconvened to develop a spiritual successor Overload, with a community level-editor and tutorial series which expands on the same ideas in thoughtful ways. What really caught my eye was how they extended the idea of “applying materials” to quad-face to assembling kit-pieces which have a modern, rather than low-poly, look. For example, rock faces have rough geometry that fill-in crevices to give those areas an organic look.

First Video in a Tutorial Series on the Overload Level Editor

Playing with these tools was a big aha moment for me – that by building-up from convex shapes there’s no need to do any kind of “decomposition” like BSPs because the space is partitioned by-construction.

I decided to use this as a starting-point and then go in my own direction.

Nightshift Galaxy Scaffold

Instead of building a completely separate application like the Overload team did, I decided to build inside the Unreal Level Editor. This is a tradeoff – on the one hand it limits the ease of offering modding tools to the community, which is something on my mind. On the other hand, though, it greatly reduced how much I needed to code (since the application-framework/asset-handling/undo systems were already in place), and let me focus on my value-adds, as opposed to re-implementing all the other loose-object handling that Unreal already does perfectly-fine.

Gamedev Streaming resumes _tomorrow_ at noon, PST. Thinking of doing a demo of the "scaffold" tool I've been making for level design, and maybe a little lecture of scene geometry in 3D. #indiegamedev pic.twitter.com/eaWQEAOSvv
— Max! (@xewlupus) April 3, 2025

Early Sneak-Peak of Scaffold running inside the Unreal Level Editor.

I won’t go into all the boring details, suffice to say for Unreal-heads out there I started with the SplineComponentVisualizer as a reference and built my own tools from there. It took about a week to get a MVP version from about 2500ish lines of code. I have all the basic operations - selection modes for vertices, edges, faces, and “cubs” (my name for convex segment – short for “cubby hole”), basic extrude and bridge options, as well as “stamps” for common shapes like cubes, rings, and cylinders.

For the grey “blockout” walls, I’m using the DynamicMesh object from the GeometryScripting extension (though I intend to replace them with instanced decoration kit meshes), and to create “real collision” trigger volumes for each cub I added a CubComponent subtype of PrimitiveComponent by referencing the implementation of ShapeComponent and BoxComponent.

I wrote a “scaffold raytracing” test which, even in it’s simple quick&dirty implementation, is 100x faster than doing ordinary raytraces with Unreal’s built-in collision. It’s not something I was use for player bullets or general-purpose traces, but for specialized batches, like enemy bullet-hell patterns, it opens up a lot of opportunities.

Post-stream: updated the BulletManager to not only use framerate-independent time, but do sub-frame position sampling for higher-precise patterns. #ScreenshotSaturday #shmup

🎶 Venus Wars OST (Joe Hisaishi) 🎶 https://t.co/otWOFaQrOE pic.twitter.com/K1Alnu0J8v
— Max! (@xewlupus) March 1, 2025

Footage from an earlier bullet-hell experimentation stream.

But the real coup de grâce for me was that, as I expected, Scaffold data also works as a navigation system! I generate a waypoint for each shared cub-face, and do a quick check to see if the “floor” of the cub is a wall-or-window, and quickly generate a navigation graph that works for both ground-units and aerial units with no extra labor.

Update from yesterday's stream -- the Scaffold level editor is now generating nav-data for AI-pathfinding in addition to blockout wall collision. #screenshotsaturday pic.twitter.com/qaUStE3V1j
— Max! (@xewlupus) April 5, 2025

First test of Aerial Navigation working inside Scaffold.

At this point I’m confident in the direction of the solution, so now I’m fleshing out missing features, and adding more “smart” operations so I can layout spaces faster. The big difference between Nightshift and Descent is that I want to exit Spaceships/Asteroid-Bases/Sewer-Tunnels/Caves/Industrial-Interiors and fly around the outside, too, so I’ll need to start thinking about a “exterior hull” toolset as well. But I want to make a half-dozen maps with validated gameplay, first.

⚙️ Tech Breakdown: AI with Finite State Machines

Mon, 18 Nov 2024 00:00:00 +0000

As I transition from developing systems to developing content, a larger ratio of my programming time is devoted to AI programming. Note: I’m discussing classical game AI – how NPCs make decisions – not the overhyped odious tech-fad of the same name.

Most recently, I coded the logic that pilots enemy jets: how to fly, when to turn, how to avoid obstacles, when to react to the player, and how to line up strafing attacks.

Music by EMMIE (Chiisana Kiseki) from the Project A-KO Greyside OST

Let’s break down the choices I made in the high-level technical design, as well as the syntax tricks I employ in low-level C++ code.

Algorithms for Game AI

The most common decision-making algorithm in games is the venerable Finite State Machine, of “FSM.” This data-structure can be easily understood by way of a “box and arrow” diagram, where each box visualizes possible States that the character can be in, and the arrows represent Transitions between those states. The characters begins in a single known initial-state, and “moves” from state to state as the associated transitions are triggered. In fact, in the Unreal Engine, character animation control is coded by authoring “Animation Blueprints” – state-machines with a box-and-arrow interface.

Sample Animation State-Machine from the Unreal Documentation

FSMs are intuitive, and easy to debug, but not without tradeoffs. The principle downside is complexity – the number of transitions grows potentially geometrically with the number of states, and can become overwealming. In Computer Science jargon, the complexity grows “O(N²) with the number of states,” and is related to a systems principle called Metcalfe’s Law.

Blueprint “Spaghettification”

The conventional answer to this problem is to break the main state-machine up into a collection of “sub-state-machines” which are individually maintainable, but require extra care and QA for their concurrent interactions.

The mid-aughts, however, were a bumper decade for devising alternatives to FSMs which didn’t suffer the complexity-explosion issue. To return to CS jaron, these generally involve creating some kind of hierarchy which limits the complexity growth to O(n × log n).

The most popular of these was the Behavior-Tree algorithm, popularized by it’s use in Halo 2, from Bungie in 2005. In a nutshell, the AI controller is continuously “descending” a tree-diagram over-and-over from the root where each branch is a condition, and each leaf is a state. When the tree evaluates to a different state than your current state, you kickoff a transition. This maintains the intuitive appeal and easily-visualized tooling of FSMs, without the complexity risk. It’s probably the most popular solution amongst FPS developers, and naturally is also available in Unreal’s “Blackboard” system.

Sample Behavior Tree from the Unreal Documentation

The main drawback here is that a carelessly constructed behavior-tree can tank-performance in the continuous-evaluation step, and most implementations often add several special-case optimizations for common bottlenecks.

An overkill algorithm I find most personally-intriguing is the Goal-Oriented Action Planning system developed for F.E.A.R., by Monolith Productions, the same year as Halo 2. This is a sophisticated planning system which marks up all the character “actions” with precondition/postcondition metadata which, combined with an elaborately-queryable environment database, allows characters to automatically “discover” action-sequences to accomplish goals without any explicit transitions. This enables enemies to “think on their feet” and often discover novel emergent solutions that surprise players and even designers. It’s even-more computationally-expensive, due to the “path-finding” aspect, but still gets a lot of use in real-time-strategy games that demand the AI to complete multi-step sequences (e.g. gather-resource -> build-factory -> attack-with-unit).

F.E.A.R. also had a great slo-mo mechanic.

All that said, for Nightshift Galaxy I ditched Unreal’s builtin stuff and followed the KISS principle and just wrote everything in C++. It’s still a mostly vanilla FSM, but with Behavior-Tree-inspired implicit transitions. Being more of an arcade-game, I’m not too worried about complexity-explosion, anyway.

C++ State Machines

In general avoiding Blueprints and developing gameplay with native C++ is win-win – not only is the performance (significantly) better, but I don’t have to deal with any “Blueprints From Hell” that require hundreds of tedious mouse-clicks to write, and are impossible to read. Furthermore, I’ve been coding C++ for decades, and have a high comfort-level with its quirks and warts. I limit Blueprints just to places where it’s required (like animation state-machines) or purely-decorative (like kicking-off VFX and SFX).

I’ve seen dozens of ways to represent FSMs in code over the years, often associating an object-per-state with several virtual-functions, but for Nightshift I’m using a simplified variant of an idea first-proposed by Charlie Tangora during my time at Yacht Club Games: one big switch! (Update from Charlie: Adam Lederer conceived it first). We declare all the states with an unnamed enum, and then shift-and-mask them together with an event-id.


/* list of states (declared in source, no header) */
enum
{
	STATE_UNDEFINED = 0,
	STATE_STANDING_STILL,
	STATE_MOVING,
};

/* list of state-events */
enum
{
	EVENT_UNDEFINED = 0,
	EVENT_STATE_ENTER,
	EVENT_STATE_UPDATE,
	EVENT_STATE_EXIT,
};

/* event structure (with per-event-type arguments) */
struct FSMEvent
{
	uint32 ID;
	union
	{
		struct { uint32 PrevID; } EnterArgs;
		struct { double DeltaTime; } UpdateArgs;
		struct { uint32 NextID; } ExitArgs;
	};

	uint64 MakeSwitch( uint32 StateID ) const
	{
		return uint64(StateID) | (uint64(ID) << 32);
	}
};

/* shorthand macro */
#define FSM_ID( SID, EID ) ( uint64(STATE_##SID) | (uint64(EVENT_STATE_##EID) << 32) )

uint32 SomeEnemy::HandleEvent( const FSMEvent& Event ) override
{
	switch( Event.MakeSwitch(State) )
	{
	case FSM_ID( STANDING_STILL, ENTER ):
	{
		/* code for entering the standing-still state */
		break;
	}
	case FSM_ID( STANDING_STILL, UPDATE ):
	{
		/* code for updating the standing-still state */
		break;
	}
	case FSM_ID( STANDING_STILL, EXIT ):
	{
		/* code for leaving the standing-still state */
		break;
	}
	case FSM_ID( MOVING, ENTER ):
	{
		/* code for entering the moving state */
		break;
	}
	case FSM_ID( MOVING, UPDATE ):
	{
		/* code for updating the moving state */
		break;
	}
	case FSM_ID( MOVING, EXIT ):
	{
		/* code for leaving the moving state */
		break;
	}
	default:
		break;
	}

	return Super::HandleEvent( Event );
}

Dispatching through a single virtual-function, rather than several per-event, was inspired by a Casey Muratori discussion with “Uncle Bob” on alternatives to “SOLID” design-patterns.

For me, there’s a lot to recommend this technique. The code is very regular and readable, and it’s easy to add, remove, or refactor states without touching any headers, which means I can still hot-recompile FSM changes while the game is running (even better than blueprints!). I can also omit empty-cases, instead of having lots of empty boilerplate handler-functions. For edge-cases that don’t naturally fit in the conventional layout, I can always add if-branches before or after the switch. A common example is using “ranges” of state IDs:


enum
{
	STATE_UNDEFINED = 0,
	STATE_IDLE,
	RANGE_AGGRO_BEGIN,
		STATE_AGGRO_TELEGRAPH_ALARM,
		STATE_AGGRO_PURSUIT,
		STATE_AGGRO_LOOK_FOR_PLAYER,
	RANGE_AGGRO_END,
};

uint32 SomeEnemy::HandleEvent( const FSMEvent& Event ) override
{
	const auto bUpdatingAggro = 
		Event.ID == EVENT_STATE_UPDATE && 
		State > RANGE_AGGRO_BEGIN && 
		State < RANGE_AGGRO_END ;
	if( bUpdatingAggro )
	{
		/* common aggro code */
	}

	return Super::HandleEvent( Event );
}

For transitions, I return a non-zero state ID from HandleEvent. This is preferable to having a SetState() subroutine like you see sometimes, because it ensures that I’m exitting the event handler function before changing the state, so our function-stack doesn’t get “plumbing” and “logic” code interleaved. Additionally, it lets me re-run the event-handler to take multiple transitions per-update, and ensure exactly one state “consumes” DeltaTime on update, which is important to avoid movement stutters (what CS peeps call “linearity”).


void SetState_Internal( uint32 InState )
{
	/* exit current state */
	FFSMEvent EvExit;
	EvExit.ID = ENEMY_EVENT_EXIT;
	EvExit.ExitArgs.NextID = InState;
	const auto ExitResult = HandleEvent( EvExit );
	/* ignore results from exit-events */
	ensure( ExitResult == 0 ); 

	/* update bookkeeping */
	FFSMEvent EvEnter;
	EvEnter.ID = ENEMY_EVENT_ENTER;
	EvEnter.EnterArgs.PrevID = State;
	State = InState;
	StateTime.SetNow( World );
	
	/* enter new state */
	if( const auto EnterResult = Enemy_HandleEvent( EvEnter ) )
	{
		/* allow enter-event to short-circuit to another state */
		SetState_Internal( EnterResult ); 
	}
}

void UpdateState( double DeltaTime )
{
	/* repeatedly update until it's consumed without transitioning */
	FFSMEvent EvUpdate;
	EvUpdate.ID = EVENT_STATE_UPDATE;
	EvUpdate.UpdateArgs.DeltaTime = DeltaTime;

	int Iter = 0; /* failsafe to avoid infinite loops */
	for(
		uint32 NewState = HandleEvent( EvUpdate );
		NewState != 0 && NewState != State && Iter < 8;
		NewState = HandleEvent( EvUpdate );
	)
	{
		SetState_Internal( NewState );
		++Iter;
	}
}

uint32 SomeEnemy::HandleEvent( const FSMEvent& Event ) override
{
	switch( Event.MakeSwitch(State) )
	{
	case FSM_ID( IDLE, UPDATE ):
	{
		if( AggroComponent->HasDetectedPlayer() )
			return STATE_AGGRO_TELEGRAPH_ALARM;
		/* ... other idle code ... */
		break;
	}
	case FSM_ID( AGGRO_TELEGRAPH_ALARM, ENTER ):
	{
		Anim->Play( AnimAlarm );
		break;
	}
	case FSM_ID( AGGRO_TELEGRAPH_ALARM, UPDATE ):
	{
		if( Anim->IsFinished( AnimAlarm ) )
			return STATE_AGGRO_PURSUIT;
		break;
	}
	case FSM_ID( AGGRO_TELEGRAPH_ALARM, EXIT ):
	{
		/* stop anim if we were interrupted for some reason */
		Anim->Clear( AnimAlarm ); 
		break;
	}
	/* ... etc ... */
	default:
		break;
	}

	return Super::HandleEvent( Event );
}

This isn’t a one-size-fits-all solution, and is largely tailored to my preferences, but it works well in practice compared to derpier things I’ve done in the past. 😅

That’s about it, lol. Thanks for reading!

⚙️ Tech Breakdown: Fixing Unreal's Horizontal Field of View

Wed, 26 Jun 2024 00:00:00 +0000

Unreal Engine applies the aspect ratio of the viewport to a fixed Horizontal Field of View. Therefore, switching from standard 16:9 to ultrawide will crop the top and bottom of the view and zoom, rather than revealing more to the left and right as you’d expect, and shrinking it to 4:3 creates an extreme fisheye effect.

Unreal Default: content on the sides of the viewport is fixed, causing weird cropping and zooming.

Our Camera: fixing content on the top and bottom of the viewport, like we expect.

The default is basically never what we want, but it’s hard-coded and there’s no way to override it with just configuration, so we have to patch it. 😖

Following the implementation of CineCamera, I opted to create a transparently-replacable subtype which does the adjustment just-in-time when the desired view is computed.

To start, we define two new subtypes: one for the actual camera component, and another for the wrapper camera actor.

/* A new camera component which overrides the default desired view */
UCLASS( meta=(BlueprintSpawnableComponent) )
class UMyCameraComponent : public UCameraComponent
{
	GENERATED_BODY()

public:

	virtual void GetCameraView(float DeltaTime, FMinimalViewInfo& DesiredView) override;
};

/* a wrapper for the camera actor which overrides the camera component subtype */
UCLASS()
class AMyCamera : public ACameraActor
{
	GENERATED_BODY()

public:
	AMyCamera( const FObjectInitializer& Init );

	UMyCameraComponent* GetMyCamera() const { return static_cast<UMyCameraComponent*>(GetCameraComponent()); }
};

Aside: The component/actor distinction is kind of annoying, because PlayerController and CameraManager handle camera components wrapped in camera actors slightly differently than camera components embedded in other kinds of actors. For CineCamera, it’s doubly-annoying because functionality is split between the two classes, so you lose features like Look at Tracking when embedding the component without the wrapper.

But I digress, for the implementation we do a little math in our override of GetCameraView and sprinkle a magic dependency-injection incantation in the wrapper’s constructor to instantiate the correct subcomponent (incidentally, this is also how you can change the default CharacterMovementComponent in subtypes of Character).

namespace CameraUtil
{

static const FName MEMBER_CameraComponent( "CameraComponent" );

static float CalcAspectRatio( UWorld* W )
{
	if( ULocalPlayer* Player = W->GetFirstLocalPlayerFromController() )
	{
		FVector2D Size( 0.0f, 0.0f );
		Player->ViewportClient->GetViewportSize( Size );
		if( Size.Y > KINDA_SMALL_NUMBER )
			return Size.X / Size.Y;
	}

	/* fallback to default when there's no viewport */
	return 16.0f / 9.0f;
}


static float GetFOVX( float Aspect, float FOVY ) const
{
	return 2.0f * R2D( FMath::Atan( FMath::Tan( 0.5f * D2R(FOVY) ) * Aspect ) );
}

static float GetFOVY( float Aspect, float FOVX ) const
{
	return 2.0f * R2D( FMath::Atan( FMath::Tan( 0.5f * D2R(FOVX) ) / Aspect ) );
}

}

/*virtual*/ void UMyCameraComponent::GetCameraView( float DeltaTime, FMinimalViewInfo& DesiredView ) /*override*/
{
	Super::GetCameraView( DeltaTime, DesiredView );

	/* early-out when we're letterboxed */
	if( bConstrainAspectRatio )
		return;

	/* for some reason, DesiredView.AspectRatio is not always right, so actually query the viewport */
	const float ActualAspect = CameraUtil::CalcAspectRatio( GetWorld() );
	const float DesiredFOVY = CameraUtil::GetFOVY( 16.0f / 9.0f, DesiredView.FOV );
	const float AdjustedFOVX = CameraUtil::GetFOVX( ActualAspect, DesiredFOVY );
	DesiredView.FOV = AdjustedFOVX;
}

AMyCamera::AMyCamera( const FObjectInitializer& Init )
	: Super( Init.SetDefaultSubobjectClass<UMyCameraComponent>( CameraUtil::MEMBER_CameraComponent ) )
{
}

Nothing much to write home about in this implementation, but I’ll call out a few details:

The Aspect Ratio applies to the Tangent of the Half-Angle of the FOV, not the angle itself, so we have to do a coordinate-space sandwich when we multiply or divide it.
I’ve wrapped FMath::RadiansToDegrees and FMath::DegreesToRadians in R2D and D2R macros because they’re annoyingly verbose for such common subroutines.
I retrieve the actual-aspect-ratio from the local player, rather than the screen, to account for Split-Screen Multiplayer (I’m assuming all split-screens have the same aspect, so we only need to look up Player 1).

If you’re really in a pinch and can’t control the subtype of CameraActor, then you can use a custom CameraModifier to post-hook the change to DesiredView with the same code. Workflow-wise this is just more annoying in the common-case, so I avoided it as the by-default solution.

Indie Game Pitching in 2024

Wed, 12 Jun 2024 00:00:00 +0000

I can’t believe how long it’s been since I wrote for the blog! My time has been fully-committed to an ambitious project to get funding to launch a new game studio.

Spoiler alert: 2024 was a bad year to try and start a studio. 😩

What follows is a post-mortem – including receipts – to pull back the curtain on this shadowy part of game development, as a service to enthusiasts and other indie developers. I also recorded a short video essay version if you just want the headlines and not all the nitty-gritty details.

Here I’ll discuss:

How game funding works
Our experience seeking funding
The state of the games industry
The future for my game, Nightshift Galaxy (TL;DR – Wishlist on Steam!)

Introduction

For new readers, hi! I’m Max! I’ve developed games professionally for twenty years. Last year I took the plunge and left my dayjob to pursue my own original indie game after some minor buzz on social media garnered inquiries from interested publishers (with help from some modest savings and my wife’s health insurance).

My ambition stirred last summer when I met up with my old college friend & roommate, Andy Jih, at a wedding. Andy started in games the same time I did, but transitioned to tech where he’s had success as an entrepreneur. I half-jokingly asked if he wanted to help me co-found a game studio. He laughed, but we started talking and a dream took shape: to found a co-op game studio and form a 6-12 person team that could make games at the so-called “Triple-I”/“Prestige” indie tier, and create a pipeline for our collaborators to pursue their own ideas.

To do this, we would need to pitch for investment.

Nerd alert! Andy and I watching anime as students, circa 2003.

How Do You Pitch a Game?

The traditional game-funding route is through publishers. These companies have capital to advance money for development costs, relationships with platform/console vendors, and marketing expertise to actually promote and sell your finished game (that last part is actually important, since after all, Nobody Knows About Your Game). In exchange, they take a percentage of the revenue. Typically, this percentage is tilted in their favor until they’ve recouped their advance, at which point the percentage is reallocated in your favor.

Submitting to publishers is easy – most of them have submission forms on their website, and accept entries from anyone. Publishers expect two things: a demo and a deck.

The demo is a preliminary prototype that establishes both your game idea and technical expertise. Developing the demo occupies a lot of your attention, but the demo is not what actually lands the deal – the goal of the demo is just to get you through the initial triage-process. It’s usually played by scouts who make recommendations, rather than by higher-up decision-makers themselves.

The deck is a powerpoint presentation which communicates the most crucial information that publishers actually need to make a decision:

What is the one-sentence elevator pitch for your game to players?
Who are you and your team, and what is your professional pedigree?
What are your game’s unique features?
What is the expected playtime?
What game engine are you using?
What is the expected size of the market?
What are comparable titles and how well did they do?
What is the target release date and platforms?
How much money are you asking for?

Usually you pitch first to the scout, who will give initial feedback and ask pointed questions. If you make a good impression, they’ll hand the materials over to a market analyst to validate the budget and revenue projections. If they like the resulting report, you’ll be called-back to pitch your deck again to actual decision-makers. If all goes well, you’ll receive a term sheet, which is their initial offer. Sometimes you discuss terms with them directly, and sometimes you hire an agent to negotiate on your behalf, but the final terms are ironed out between lawyers to produce a final contract. Sign that, and the advance is deposited in your bank account. Then you make the game!

If the publisher-route isn’t a good fit, there are other options that can replace or supplement a publishing deal:

Project Financing just deals with budget advances and leaves it up to you to market and port the game. This makes more sense for mature studios that have in-house marketing teams and an existing pipeline of game projects rather than startups like us.
Equity Funding invest in the studio itself, rather than a specific project, with the goal of either seeing a long revenue tail (e.g. live-service games) or the opportunity to sell their stake to a larger studio once your “brand” has value.
Angel / Pre-Seed Funding is a fancy way of saying “hitting up rich friends for several small good-faith investments.”

We targeted the publisher route, but also talked to some equity and pre-seed investors. The first order of business was to prep our pitch.

Creating the Demo

I transitioned from experimental prototyping to creating a self-contained demo in August and ultimately took three months to get a build ready. This time was spent validating that the gameplay was ready for pick-up-and-play by participating in public playtesting through Glitch City LA and reacting to real play-sessions by strangers. Additionally, I extensively tested and fixed all the bugs in the code, and did basic optimizations to ensure it ran at a smooth 60fps on a Steam Deck. You can’t babysit it once you submit it, so it just has to work.

Lastly, I integrated samples of polished artwork and music to better represent the look and feel of the final product. For those assets, we used a mixture of generic asset-store models and called up favors from friends in the industry who agreed to well-below their normal rates, for which I’m eternally grateful!! Shout-outs to Jack Covell, Troupe Gammage, Cathryn Stark, Meredith Alden-Lewis, Cherie Heiberg, Issa, and Evan Palmer.

In terms of playtime, a casual playthrough of the demo lasts 10-15 minutes.

In total, we ended up spending about $5,000 on models, illustrations, sound, and music for the demo and deck (actually a huge discount!!).

Testing the demo at Glitch City’s “BYOG(ame)” night.

To help build publicity for the project at this point, I also began regularly dev-streaming demo work on Twitch.

My cat, The Boy, is a regular guest on our weekly dev stream.

Response to the demo was warm enough that we were invited to present with the Media Indie Exchange at Los Angeles Comic Con. The reception was overwhelmingly positive, including a great playthrough by Alex Wilmer and Bannon Rudis, and gave us a lot of confidence in moving forward.

Very positive reception from very diverse players.

Special thanks to Meredith and Cherie, my wi~ife, for helping mind our booth.

Creating the Deck

For the deck, I focused on the first half, describing the game, and Andy worked on the second half, presenting the market analysis and budget. We reached out to our industry contacts for about a dozen test-pitches, refining the slides as we received feedback. We ultimately spent over two months creating the final deck, which had about 30 total slides.

The hardest part was actually coming up with the name of the game, haha. We actually made a giant google doc of random ideas, and it was Cherie who suggested Nightshift Galaxy, combining references to gig work, science fiction, and transforming robots.

Work-in-progress thumbnails from early-on developing the deck.

Final Presentation Quality, courtesy of Evan Palmer.

A few selected slides.

In the interest of full context and radical transparency, I’ll share all the headline details from our original pitch:

Target: PC, Steam Deck, and all consoles (Unreal Engine 4)
Single Player + Online Co-op
Development schedule: 2.5 Years
Team size: 6 full-time employees, 8 part-time contractors
Budget ask: $3 million

I’m just going to take a step back and say that, for me, 3 million dollars is a staggering amount. I come from Rust belt Ohio where that could be a comfortable middle-class person’s total lifetime income. The easiest way to wrap your head around costs like that is the following rule of thumb: estimate $10k/head/month (Note: his doesn’t mean everyone is making over 100k/year, because it’s including health insurance, personnel costs, overhead, contingency, etc). So if we count 2 part-time people as 1 “full time” person, you get:

10 people × 30 months × $10k = $3mm

Andy prepared a detailed spreadsheet with itemized costs, and it came out to that same ballpark. From our test pitches, we’d heard feedback that mid-size indie games range from $1 million to $10 million, so we were feeling pretty good that we were at the “bottom third” of that range, which felt right.

We picked comps (“comparables”) that reflected a market-appetite for mecha combat and arcade-style flight sims, but with an opportunity to come in with a colorful and lighthearted anime tone, rather than grimdark military-industrial-complex-mercenary stuff.

Pitching at Conferences

Despite having early versions of the pitch ready by December, we heeded advice to wait until Q1 2024 to start pitching so we don’t get lost in the holiday season shuffle. Furthermore, there are two industry conferences, DICE in Las Vegas in late February and GDC in San Francisco in early March, which are natural times to schedule face-to-face meetings with publishers and investors. These can be scheduled using a (paid) service called MeetToMatch.

In the meantime, I continued to promote the project with sizzle footage on social media and Andy arranged finding a lawyer and incorporating a C-Corp, tinygalactic Inc, so we could hit the ground running once we entered into negotiations with a publisher or investor, as well as offer concrete equity terms to partners though a co-op policy Andy designed.

Also, reckoning that we could get a term sheet before summer, Andy made the rounds soliciting $100k of small commitments from a friends-and-family round of angel investment as a financing bridge to start hiring full-time partners in anticipation of a publisher advance.

Lastly, feeling confident about our demo’s reception at the MIX, we applied to Day of the Devs, a prestigious indie-game show during GDC founded by Double Fine, and were greatly surprised to be accepted, despite being so early in pre-production. Consequently, we needed to hurriedly create a Teaser Trailer for their stage reel. I mostly captured in-game footage myself, and then we reached out to a friend-of-a-friend to edit it together.

Sleep deprivation mania.

🔥 Demoing at Day of the Devs. 🔥

All told, the cost of legal fees, software licenses, demo equipment, conference registrations, travel, and video post-production came out to about another $5,000 for a total out-of-pocket at this point of about $10,000 (not counting the opportunity cost of, you know, not having jobs).

Between Zoom calls that we set up before the conferences, the conferences themselves, and several weeks of followups we gave the pitch about two dozen times, we met with several other indies pitching at about the same range who were all generous enough to share their experiences.

Having a playable demo at our fingertips was big for guerilla encounters.

We sent off a few early online submissions before we took off for DICE, where we started to get our first early-warning feedback: the industry was in a downturn and places that used to regularly sign $5mm deals weren’t considering anything north of $1.5 million, and many places were capping out at $500k.

Andy and I had several emergency meetings and determined that by dropping online-multiplayer and halving the amount of content we could get by with 2/3rds the staff, and get the budget-ask down to just below $2 million, so we revised our deck to reflect that before sending out our second round of submissions and taking off for GDC. We had a hard time going lower than that without just fundamentally changing the game for which we’d already built a demo.

The “excitement” of publisher speed dating.

The highly positive responses we received in meetings and demos encouraged us greatly, including a wonderful interview and playthrough with the Kinda Funny podcast crew, so we came home very optimistic.

Big thanks to Andy Cortez and Blessing Adeoye for a great interview!

“Survive to 2025”

All told, we submitted our pitch to over 50 publishers. 15 of them haven’t responded, which we’ve heard is quite common. Of the 35 who have responded, there were a few who said that our game didn’t quite fit their wheelhouse (they’re focused on edgier games, for example). The majority of publishers, though, gave very positive feedback on the game itself, including excited personal notes and even descriptions of detailed playthroughs with design ideas, but uniformly took a poor financial outlook for comps. The market is too overcrowded (almost 15K games were released on Steam in 2023, alone) to justify writing checks over $500K, and could we kindly come back with a dramatically lower budget?

Calling around to other devs, we’ve learned that the story is the same for everyone right now. Nobody is investing in mid-range indie games, even people with more mature pitches than ours. Just really small and really big games. This point was articulated poignantly in this noclip discussion on their own struggles to find funding. Coupled with rampant downsizing in the industry and console vendors not signing any exclusives, things are dire for game developers.

However, with no clear term sheet on the horizon and because Andy had reached the end of his financial runway and needed to find a job, this proved fatal for the dream of tinygalactic Inc.

As a consequence, and with a heavy heart, I no longer felt comfortable pitching a multimillion-dollar game studio on my own, so we let our prospective partners know they should accept their other job opportunities and politely let our angel investors know that we were no longer seeking funding.

Future for Nightshift Galaxy

While we’re not founding a new game studio, I’m going to continue working on it as a solo developer. I know it’s a great game that excites me, and I feel responsible to the followers I’ve been juicing up with sizzle footage for months.

To demonstrate my commitment, I have finally launched the Steam page so you can add it to your wishlist. My next milestone is to develop the build enough to enter Early Access so people can begin to download and play the demo themselves.

I do need to scope the game down a lot to be doable without a whole team, however, so I’m in the process of redesigning it to focus specifically on action and gameplay. While I’ll still post sizzle footage from time-to-time, I hope to primary resume regular log-form blogging here as I work through it.

Let’s goooo!

The possibility is still open with a few publishers where a deal can be made for a small polish-for-release budget, and funding opportunities might be better next year, but in the meantime, if you’d like to help me pay my bills, I’ve opened up a Ko-Fi tip jar.

The Boy remains on the team!

Thank you for reading, and I look forward to having more to share here, soon.

👷 Project Update: Tiny Starfighter

Tue, 04 Jul 2023 00:00:00 +0000

Hey everyone, sorry for the posting delay, but I’ve been focussed on my game for the last two months, and now I have a demo build, which I made for Glitch City LA x GUMBO NYC!

Player Mechanical Design by Evan Palmer.

I’m still working on Part 3 of the Locomotion Series, and hope to resume regular posting soon, as well as reveal other exciting project updates!

⚙️ Procedural Animation: Locomotion (Part 2)

Tue, 02 May 2023 00:00:00 +0000

In Part 1 we took care of the preliminaries, setting up our assets, and getting a base layer root and hip motion going on which to start grooving. In this part we’ll take the next step and start stepping.

One of the famous Eadweard Muybridge photos.

First things first, to simplify things, let’s refactor our code for the legs into helper structs so that we don’t have to copy-paste each line for both the left and right legs.

struct FootController
{
	FVector RestPos;

	bool IsLeft( const ALocoPawn& Pawn ) const { return this == &Pawn.FootL; }
	FVector& RefPos( ALocoPawn& Pawn ) const { return IsLeft( Pawn ) ? Pawn.FootPosL : Pawn.FootPosR; }

	void Init( ALocoPawn& Pawn );
	void Update( ALocoPawn& Pawn, float DeltaTime );
};

FootController FootL;
FootController FootR;

// LocoPawn.cpp

void ALocoPawn::FootController::Init( ALocoPawn& Pawn )
{
	RestPos = RefPos( Pawn );
}

void ALocoPawn::FootController::Update( ALocoPawn& Pawn, float DeltaTime )
{
	const FTransform& ToWorld = Pawn.GetActorTransform();
	FVector& Pos = RefPos( Pawn );
	Pos = ToWorld.TransformPosition( RestPos );
	Pos.Z = 0.0f;
	Pos = ToWorld.InverseTransformPosition( Pos );
}

Rather than trying to fully puzzle-out stepping all at once, I thought I’d start with the simplest idea: we already implemented hip-swaying last week. What if we just pick up our feet in sync with the sway?

UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float StepHeight = 75.0f;

// LocoPawn.cpp

void ALocoPawn::FootController::Update( ALocoPawn& Pawn, float DeltaTime )
{
	const FTransform& ToWorld = Pawn.GetActorTransform();
	FVector& Pos = RefPos( Pawn );

	float RadiansOffset = IsLeft( Pawn ) ? 0.0f : PI;
	float Radians = FMath::Fmod( 0.5f * Pawn.HipPhase * 2.0f * PI + RadiansOffset, 2.0f * PI );
	float Arc = FMath::Max( 0.0f, Pawn.HipMulti * FMath::Sin( Radians ) );

	Pos = ToWorld.TransformPosition( RestPos );
	Pos.Z = Pawn.StepHeight * Arc;
	Pos = ToWorld.InverseTransformPosition( Pos );
}

We take the Max of the of the sway oscillation and 0 to ensure that the foot is flat at Z=0 during the off-cycle.
We stagger the right foot by adding PI (180 degrees) to its phase.

To be honest, my expectations were low, but this doesn’t look too bad. ⛸️

There’s an obvious flaw with this version: the feet are sliding on the ground like ice-skates. But there’s also a subtle flaw: the feet are making contact directly-below the body, as opposed to out in front. So we need to make two changes:

Pin the foot to world-space when in contact with the ground.
Extrapolate the target step position ahead of the body.

UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float StepExtrap = 0.15;

struct FootController
{
	...

	FVector PinnedPos;
	bool bPinned;
};

// LocoPawn.cpp


void ALocoPawn::FootController::Init( ALocoPawn& Pawn )
{
	...

	PinnedPos = ToWorld.TransformPosition( RestPos );
	bPinned = true;
}

void ALocoPawn::FootController::Update( ALocoPawn& Pawn, float DeltaTime )
{
	...

	Pos = ToWorld.TransformPosition( RestPos );
	Pos += Pawn.StepExtrap * Pawn.Velocity; // offset step-target out-front
	Pos.Z = 0.0f;

	// pin status changed?
	bool bWantPin = Radians >= PI;
	if( bPinned != bWantPin )
	{
		if( bWantPin )
			PinnedPos = Pos;
		else
			PinnedPos = ToWorld.InverseTransformPosition( PinnedPos );
		bPinned = bWantPin;
	}

	// apply current status
	if( bPinned )
	{
		Pos = ToWorld.InverseTransformPosition( PinnedPos );
	}
	else
	{
		float X = 0.5f - 0.5f * FMath::Cos( Radians );
		Pos.Z += Pawn.StepHeight * Arc;
		Pos = FMath::Lerp( PinnedPos, ToWorld.InverseTransformPosition( Pos ), X );
	}
}

I chose Cos(Radians) for the horizontal stepping interpolation, because the height is Sin(Radians), so that would produce a circular arc, which I thought might look good?

STOMP! STOMP! STOMP! 🥾

So circular arcs produce a stomping gait, because the end of the motion is straight-down into the ground. But when we’re walking normally, we’re usually straighening out our knee on the descent, causing our feet to glide into contact with the ground, like an airplane landing.

To translate this to code, I noodled around with the Desmos Graphing Calculator until I found a StepArc graph which matches our animation onion-skin, on the normalized [0-1] range.

I can’t quite explain the art of designing graphs; I’ve just been praticing since I first played Green Globs in middle school. In this case I was combining an up-down arc with an ease-out curve.

float StepArc( float X )
{
	// simplified version of the graph we designed
	float _X = ( 1.0f - X );
	return 9.481481481f * _X * _X * _X * X;
}

void ALocoPawn::FootController::Update( ALocoPawn& Pawn, float DeltaTime )
{
	...

	{
		float X = FMath::Min( 1.0f, Radians / PI ); 
		Pos.Z += Pawn.StepHeight * Pawn.HipMulti * StepArc( X );
		Pos = FMath::Lerp( PinnedPos, ToWorld.InverseTransformPosition( Pos ), X );

	}
}

Smooth as silk! 🦵

We could of course interpolate the two step arcs with a control parameter called stompiness if our game design calls for it.

This looks great, so that’s a good place to leave it for now, but one last piece of foreshadowing: to get results looking this nice, I did have to dial-down the movement speed by more than half. The root position was cruising at sprinting speeds before, and the step-frequency required to keep up resembled a Benny Hill sketch.

Cue the orchestra.

The reason is because our whole mental model up until this point has describe how people walk, but not how they run. In Part 3 we’ll revise our mental model and introduce changes at each layer to enable sprinting. Thank you for reading! 💖

Full Sample Code for Part 2 here.

⚙️ Procedural Animation: Locomotion (Part 1)

Sat, 29 Apr 2023 00:00:00 +0000

This is the first article in a series on implementing procedural locomotion. I’m going to walk through step-by-step generating walkcycles for a pair of robo-pants. Part 1 will mostly be setup and preliminaries, but we’ll at least go far enough for the gestures to start exihibiting personality.

Download the model, if you’d like to follow along.

Character locomotion in games is typically two-step process: first we simulate the position of the character component in the world (sometimes called the root or capsule position), and then we blend and advance animations to match that movement with authored-clips in the character’s local component-space.

For polished 3D games, this is a labor-intensive process, because animators must provide complete animation sets (Idle, Walk, Jog, Run, Jump, etc), as well as various layered, additive, and interstitial animations to fill in corner-cases. Subsequently, indie devs on a budget are always seeking ways to automate these animations.

David Rosen’s 2014 GDC talk on animations in Overgrowth is the best presentation on this subject. He describes a method of generating animations from a small set of key-poses.

Because I don’t have an “animation team”, and I want to support a character creator where players can kitbash together new mecha in-game, I’ve been experimenting with generating animations synthetically with just code.

Kitbashed-character animation tests.

I’m taking direction from Richard Williams’ indispensible textbook, The Animator’s Survival Kit, which I highly recommend reading if you’re interested in animation at all.

Every time I re-read it I learn something new.

Let’s get started with a simple test Pawn in Unreal which just moves around on a flat plane (let’s not worry about collision or anything). Here’s our starting-code:

#pragma once
#include "GameFramework/Pawn.h"
#include "LocoPawn.generated.h"

UCLASS( Abstract )
class ALocoPawn : public APawn
{
	GENERATED_BODY()

public:

	/* Components */
	UPROPERTY( VisibleAnywhere, BlueprintReadOnly )
	class USkeletalMeshComponent* Skel;	

	/* Movement Configuration */
	UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
	float MovementSpeed = 1000.0f;
	UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
	float AccelSeconds = 0.5f;
	UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
	float DecelSeconds = 0.25f;
	UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
	float RotationSeconds = 0.1f;
	UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
	float CameraSpeed = 90.0f;

	ALocoPawn();

	/* APawn Interface */
	virtual void SetupPlayerInputComponent( UInputComponent* Comp ) override;
	virtual void BeginPlay() override;
	virtual void Tick( float DeltaTime ) override;

private:

	FVector Position, Velocity = FVector( 0.0f ), Accel = FVector( 0.0f );
	float TargetYaw, Yaw, YawSpeed = 0.0f;
};

// LocoPawn.cpp
#include "LocoPawn.h"
#include "Components/SkeletalMeshComponent.h"

static const FName NAME_MoveX( "MoveX" );
static const FName NAME_MoveY( "MoveY" );
static const FName NAME_CamX( "CamX" );

void DampSpring( float& Value, float& Speed, float Target, float Duration, float DeltaTime )
{
	const float Omega = 2.0f / Duration;
	const float Exp = FMath::Exp( -Omega * DeltaTime );
	const float Change = Value - Target;
	const float Temp = ( Speed + Change * Omega ) * DeltaTime;
	Speed = ( Speed - Temp * Omega ) * Exp;
	Value = Target + ( Change + Temp ) * Exp;
}

ALocoPawn::ALocoPawn()
{
	static const FName NAME_SkelComponent( "SkelComponent" );
	Skel = CreateDefaultSubobject< USkeletalMeshComponent >( NAME_SkelComponent );
	RootComponent = Skel;
}

/*virtual*/ void ALocoPawn::SetupPlayerInputComponent( UInputComponent* Comp ) /*override*/
{
	Super::SetupPlayerInputComponent( Comp );
	Comp->BindAxis( NAME_MoveX );
	Comp->BindAxis( NAME_MoveY );
	Comp->BindAxis( NAME_CamX );
}

/*virtual*/ void ALocoPawn::BeginPlay() /*override*/
{
	Super::BeginPlay();

	/* Initialize Movement */

	Position = GetActorLocation();
	Position.Z = 0.0f;
	Velocity = FVector( 0.0f );
	Yaw = TargetYaw = GetActorRotation().Yaw;

	SetActorLocationAndRotation( Position, FRotator( 0, Yaw, 0 ) );
}

/*virtual*/ void ALocoPawn::Tick( float DeltaTime ) /*override*/
{
	Super::Tick( DeltaTime );

	/* Update Camera Rotation */

	AddControllerYawInput( DeltaTime * CameraSpeed * GetInputAxisValue( NAME_CamX ) );

	/* Update Root Movement */

	FRotator ControlRotation ( 0, GetControlRotation().Yaw, 0 );
	FVector Stick( GetInputAxisValue( NAME_MoveY ), GetInputAxisValue( NAME_MoveX ), 0.0f );
	Stick = ControlRotation.RotateVector( Stick ).GetClampedToMaxSize( 1.0f );
	float StickTilt = Stick.Size();

	Accel = FVector( 0.0f );
	if( StickTilt > 0.35f )
	{
		float AccelFriction = 4.0f / AccelSeconds;
		Accel -= AccelFriction * Velocity;
		Accel += AccelFriction * MovementSpeed * Stick;
		TargetYaw = FMath::RadiansToDegrees( FMath::Atan2( Stick.Y, Stick.X ) );
	}
	else
	{
		float DecelFriction = 4.0f / DecelSeconds;
		Accel -= DecelFriction * Velocity;
	}
	Velocity += Accel * DeltaTime;
	Position += Velocity * DeltaTime;
	
	float NormalizedSpeed = FMath::Min( 1.0f, Velocity.Size() / MovementSpeed );
	if( NormalizedSpeed > 0.05f )
	{
		float DeltaAngle = FRotator::NormalizeAxis( TargetYaw - Yaw );
		DampSpring( Yaw, YawSpeed, Yaw + DeltaAngle, RotationSeconds, NormalizedSpeed * DeltaTime );
	}

	SetActorLocationAndRotation( Position, FRotator( 0, Yaw, 0 ) );
}

Nothing really remarkable going on here, though I’ll highlight a few points of interest:

The DampSpring method is the same one I described in my article on Trailing Joints.
The 4/T approximation for the coefficient of friction has always been a nice rule-of-thumb.
Using NormalizedSpeed to scale the DeltaTime argument in the rotation update is pretty clever, if I do say so myself 😎

With this in hand, we can make a child Blueprint which sets the appropriate assets.

I also added a camera on a spring-arm, just so we don’t have to code that up.

We can test at this point and confirm that we have some basic movement and camera controls, but the pants are pretty stiff since there’s not animation yet. We’ll proceed by adding secondary movement, layer by layer.

To start designing effects, we need a mental model of how locomotion actually works. Taking a cue from Williams, I’m thinking about it like this:

We start walking by leaning forward, initiating a fall.
We throw one of our feet ahead to catch ourselves (the “contact” keyframe)
We then push ourselves back-up, while picking up our back foot (the “passing” keyframe)
Rinse & Repeat.

So let’s start by “leaning” into our acceleration. The cross-product of up with the acceleration produces the correct rotation axis for the lean, so I started with that. I noticed it was a bit too snappy, though, so I smoothed it out with a low-pass filter (FDampedQuat is just a helper struct I wrote to apply DampSpring to a rotation).

/* Leaning Configuration */
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float LeanMulti = 0.64f;
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float MaxLeanAngle = 45.0f;
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float LeanSmoothingSeconds = 0.25f;

FDampedQuat SmoothLean;

// LocoPawn.cpp

/*virtual*/ void ALocoPawn::Tick( float DeltaTime ) /*override*/
{
	...

	FQuat TargetLean = FQuat::Identity;

	// ^ is a shorthand for cross-product in Unreal
	FVector Lean = FVector::UpVector ^ Accel;

	float LeanAmount = Lean.Size();
	if( LeanAmount > 1.0f )
	{
		FVector LeanAxis = Lean / LeanAmount;
		float LeanAngle = LeanMulti * ( LeanAmount / 100.0f );
		LeanAngle = FMath::Min( LeanAngle, MaxLeanAngle );
		TargetLean = FQuat( LeanAxis, FMath::DegreesToRadians( LeanAngle ) );
	}

	SmoothLean.Update( TargetLean, LeanSmoothingSeconds, DeltaTime );

	// note how we multiple on the left because the lean is in world-space
	FQuat Rotation = SmoothLean.Value * FRotator( 0, Yaw, 0 ).Quaternion();

	SetActorLocationAndRotation( Position, Rotation );
}

Already our character is looking more physical, and even catches itself when it stops 🤩

Normally I’d be more subtle, but I’ve exagerrated it for demonstration purposes.

At this point we need to start rotating joints, so let’s set up an Animation Blueprint. For this tutorial, I’m just using out-of-the-box AnimNodes that are built into Unreal. If you want to know more about Two Bone IK, you can read my breakdown.

My convention of stacking blueprint boxes isn’t idiomatic, but I prefer it because having statements top-to-bottom and expressions left-to-right matches how code looks in text.

This blueprint has four public parameters (seen on the bottom-elft in the image):

LeftFootPos - the position of the left foot (in component-space)
RightFootPos - the position of the right foot (in component-space)
HipOffset - an adjustment to add to the hip location
HipRotation - an adjustment to add to the hip rotation

These are applied in the pipeline on the right: first we start with the mesh-space ref pose. If you have “key-poses”, like in Overgrowth, you could hook those in here. I did that on Solar Ash to give procedurally-animated level-scale characters “stances” before the IK solvers adjusted them to conform to the environment.

Next we adjust the hips with a Modify-Bone node (I’ve hidden pins we don’t need). Lastly, we use the built-in IK nodes rotate the legs joints so the feet end up in the right place.

You can test these parameters but adjusting them in the Anim Preview Editor panel, and observing the character pose in the scene preview panel (wiggling HipOffset.Z makes a little dance, which I can never resist trying).

To drive the animation from C++, we have to add some output parameters, and then “hook them together” in the Blueprint’s Update.

/* Outputs to Animation Instance */
UPROPERTY( BlueprintReadOnly )
FVector FootPosL = FVector( 0, -30, 0 );
UPROPERTY( BlueprintReadOnly )
FVector FootPosR = FVector( 0, 30, 0 );
UPROPERTY( BlueprintReadOnly )
FVector HipOffset = FVector( 0, 0, 0 );
UPROPERTY( BlueprintReadOnly )
FRotator HipRotation = FRotator( 0, 0, 0 );

This seems like boilerplate. Why do we have to declare everything twice?

The technical reason is that animations in Unreal evaluate on background threads in parallel and can’t access unsynchronized main-thread data, so we give them their own copy. It also lets multiple unrelated Actors share the same animation setup.

But the real virtue of this system becomes clear when you work in a big team: the Gameplay Programmers who maintain the Pawn and the Technical Animators who maintain the Animation Blueprint can keep their work separate, but use this update function as a “handshake” where you link up what the game provides with what the animation needs.

Anyway, let’s test this by adjusting the feet so they don’t leave the ground when the character leans (In a real game you’d do raycasts for this).

FVector FootRestPosL, FootRestPosR;

// LocoPawn.cpp

/*virtual*/ void ALocoPawn::BeginPlay() /*override*/

	...

	/* Initialize feet */

	FootRestPosL = FootPosL;
	FootRestPosR = FootPosR;
}


/*virtual*/ void ALocoPawn::Tick( float DeltaTime ) /*override*/
{
	...

	// reset feet to rest-position
	FootPosL = FootRestPosL;
	FootPosR = FootRestPosR;

	// convert feet to world-space
	FTransform ToWorld( Rotation, Position );
	FootPosL = ToWorld.TransformPosition( FootPosL );
	FootPosR = ToWorld.TransformPosition( FootPosR );

	// snap feet to ground
	FootPosL.Z = 0.0f;
	FootPosR.Z = 0.0f;

	// convert feet back to component-space
	FootPosL = ToWorld.InverseTransformPosition( FootPosL );
	FootPosR = ToWorld.InverseTransformPosition( FootPosR );

}

Nice little knee-bends while turning.

The last thing we’ll do in this first part is to begin giving the hips some motion. Checking back in with Williams’ textbook, we see his basic walkcycles moving the hips like a sine-wave, so let’s try that.

/* Hip Configuration */
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float HipPhaseSpeed = 2.0f;
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float HipAmplitudeDampSeconds = 0.5f;
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float HipOffsetZ = 20.0f;
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float HipBiasZ = -17.0f;
UPROPERTY( EditDefaultsOnly, BlueprintReadWrite )
float HipRoll = 8.0f;

float HipPhase = 0.0f;
float HipMulti = 0.0f;
float HipMultiSpeed = 0.0f;

// LocoPawn.cpp

/*virtual*/ void ALocoPawn::Tick( float DeltaTime ) /*override*/
{
	...

	DampSpring( HipMulti, HipMultiSpeed, StickTilt, HipAmplitudeDampSeconds, DeltaTime );
	HipPhase += HipPhaseSpeed * NormalizedSpeed * DeltaTime;
	HipOffset.Z = HipMulti * ( HipBiasZ + HipOffsetZ * FMath::Sin( HipPhase * 2.0f * PI ) );
	HipRotation.Roll = HipMulti * HipRoll * FMath::Sin( 0.5f * HipPhase * 2.0f * PI );
}

I modulated the hip movement based on how much the stick is tilted (passed through a low-pass filter, first, like the lean). Then advanced phase of the sin wave based on the NormalizedSpeed so it’s higher-frequency when we’re fast, and lower-frequency when we’re slow. Lastly, I swayed the hips with a roll at half the frequency, so there’s a left-right-left-right cadence as it oscillates.

Now we’re starting to get some personality!

That’s a good place to stop for today. You can find a full code listing for Part 1 here. Next time we’ll talk about taking steps.

⚙️ Procedural Animation: Trailing Joints

Thu, 16 Mar 2023 00:00:00 +0000

This is my second article on procedural animation. This time I’ll break down a simple “trailing” effect, for joints which “hang loose” and trail behind the primary movement. This is a subtle effect which can create a physical feel without too much actual physics.

Wings trailing behind the character to create secondary motion.

I use this to produce layered secondary motion out of otherwise simple base animations. Many of them, like the dash, are just single-frame poses!

Limbs also trail when dashing, to avoid looking stiff. All the motion here is coming from trailing and headlook!

As with IK, Unreal has some okay builtins to do this stuff, but I like writing my own so I can fine-tune them. I’ll be using Unreal C++ snippets for my explanation, but I’ll include a Unity C# listing at the bottom of the article.

Here’s the basic outline of my solution: we configure a relative “rest offset” from a “joint” bone where a “particle” sits when the system is at rest. When the joint moves, an “error” offset is induced between the particle and the target offset, which the particle resolves with a simulation, creating the trailing motion.

(1) The sytem at rest, (2) the joint offset, (3) the particle simulation.

Two-point systems like this only approximate rotational dynamics, but that’s fine! A “good enough” solution that’s simple and easy to configure is more useful than a “scientifically correct” simulation that’s full of weird, hard-to-pick coefficients. Games are theater!

To add a joint animation effect in unreal you need to implement the FAnimNode interface. I’m subtyping FAnimNode_SkeletalControlBase which takes care of most of the boilerplate.

USTRUCT( BlueprintInternalUseOnly )
struct FAnimNode_SimpleTrail : public FAnimNode_SkeletalControlBase
{
	GENERATED_BODY()

	// Configuration
	UPROPERTY(EditAnywhere, Category = Solver)
	FBoneReference JointBone;       // which bone will act at the "joint"
	UPROPERTY(EditAnywhere, Category = Solver)
	FVector LocalTarget;           // the relative offset from the joint to the target-position
	UPROPERTY(EditAnywhere, Category = Solver)
	float DampSeconds = 0.2f;      // the time it takes for the particle to resolve the target
	UPROPERTY(EditAnywhere, Category = Solver)
	float AngleLimit = 30.0f;      // the maximum allowed angle, in degrees

	// Simulation State
	FVector ParticlePos;     // the "particle" position
	FVector ParticleVel;     // the "particle" speed
	float DeltaTime = 0.0f;  // how many seconds to advance the simulation next?
	bool bReset = true;      // should we "reset" the simulation?

	// FAnimNode interface
	virtual bool NeedsDynamicReset() const override;
	virtual void InitializeBoneReferences( const FBoneContainer& RequiredBones ) override;
	virtual bool IsValidToEvaluate( const USkeleton* Skeleton, const FBoneContainer& RequiredBones ) override;
	virtual void ResetDynamics( ETeleportType InTeleportType ) override;
	virtual void UpdateInternal( const FAnimationUpdateContext& Context ) override;
	virtual void EvaluateSkeletalControl_AnyThread( FComponentSpacePoseContext& Output, TArray<FBoneTransform>&  OutBoneTransforms) override;
};

Adjusting LocalTarget tunes how much rotation is induced by movement. A smaller value will have a more extreme response to small movements, and a large value will only respond to large movements. Adjusting DampSeconds tunes how quickly the rotation returns to the rest pose (less time = stiffer feel).

I always prioritize picking configuration units people can understand, like distance and time, rather than “multipliers” or “fudge factors.”

The important method is EvaluateSkeletalControl_AnyThread which takes a pose as an input, and writes a list of bone transforms back to the animation system. We’ll breeze past the other methods quickly, they’re not interesting:

bool FAnimNode_SimpleTrail::NeedsDynamicReset() const
{
	// register for a callback when we need to "restart" the simulation (e.g. due to aniatmion state changes)
	return true;
}

void FAnimNode_SimpleTrail::InitializeBoneReferences( const FBoneContainer& RequiredBones )
{
	// map the bone name to a bone-index on a particular skeleton
	JointBone.Initialize( RequiredBones );
}

bool FAnimNode_SimpleTrail::IsValidToEvaluate( const USkeleton* Skeleton, const FBoneContainer& RequiredBones )
{
	// confirm our skeleton actually has the joint with the right name and that
	// our local target is nonzero (otherwise we can't convert the offset to a rotation)
	return JointBone.IsValidToEvaluate( RequiredBones ) && LocalTarget.SizeSquared() > 1e-4f;
}

void FAnimNode_SimpleTrail::ResetDynamics( ETeleportType InTeleportType )
{
	// set a flag for evaluate to use
	bReset = true;
}

void FAnimNode_SimpleTrail::UpdateInternal( const FAnimationUpdateContext& Context )
{
	// save the frame time for the simulation to consume
	Super::UpdateInternal( Context );
	DeltaTime = Context.GetDeltaTime();
}

Note that we’re often dealing with SizeSquared(), rather than Size() directly, because it’s cheaper to compute.

Before we write EvaluateSkeletalControl_AnyThread lets make a helper function which does the simulation logic. It’s actually really useful for other things, too! You really don’t need to understand the math to use it, though if you’re interested it implements the closed-form solution to the critically-damped spring equation.

void DampSpring( FVector& Pos, FVector& Vel, const FVector& Target, float DampSeconds, float DeltaTime )
{
	float Omega = 2.0f / Duration;
	float Exp = FMath::Exp( -Omega * DeltaTime );
	FVector Change = Value - Target;
	FVector Temp = ( Vel + Change * Omega ) * DeltaTime;
	Vel = ( Vel - Temp * Omega ) * Exp;
	Pos = Target + ( Change + Temp ) * Exp;
}

I originally encountered this function in Game Programming Gems 4.

Now we’re ready to evaluate the joint swing. We start by extracting the current bone transform in component-space (“CS”), and converting it to world-space (“WS”), and using that to compute our particle target.

void FAnimNode_SimpleTrail::EvaluateSkeletalControl_AnyThread( FComponentSpacePoseContext& Output, TArray<FBoneTransform>&  OutBoneTransforms )
{
	FBoneContainer& RequiredBones = Output.Pose.GetPose().GetBoneContainer();
	FCompactPoseBoneIndex BoneIndex = JointBone.GetCompactPoseIndex( RequiredBones );
	FTransform InBoneCS = Output.Pose.GetComponentSpaceTransform( BoneIndex );
	FTransform ToWorld = Output.AnimInstanceProxy->GetComponentTransform();
	FTransform InBoneWS = InBoneCS * ToWorld; // note that FTransform * order is transposed
	FVector ParticleTarget = InBoneWS.TransformPositionNoScale( LocalParticleTarget );

With the target in hand, we have everything we need to update the simulation (note that 1e-5f is just scientific-notion for 0.00001 i.e. “a very small number”).

	if( bReset ) // a reset was requested -- just snap to the target
	{
		ParticlePos = ParticleTarget;
		ParticleVel = FVector::ZeroVector;
		bReset = false;
	}
	else if( DeltaTime > 1e-5f ) // advance the simulation by DeltaTime
	{
		DampSpring( ParticlePos, ParticleVel, ParticleTarget, DampSeconds, DeltaTime );
		DeltaTime = 0.0f;
	}

With the particle and target positions we can compute the “swing” rotation from the target-offset to the particle-offset.

	FVector TargetOffset = ParticleTarget - InBoneWS.GetLocation();
	FVector CurrentOffset = ParticlePos - InBoneWS.GetLocation();
	FQuat Swing = FQuat::FindBetween( TargetOffset, CurrentOffset );

In order to constrain the swing to AngleLimit we have to decompose the quaternion into its Angle and Axis components. Unreal’s ToAxisAndAngle method does this for us.

	FVector SwingAxis;  // the direction of rotation in 3D, like the axle of a wheel
	float SwingRadians; // the "mathy" unit of rotation. pi radians = 180 degrees
	Swing.ToAxisAndAngle( SwingAxis, SwingRadians );

Now we check the angle, and if it’s too big, we adjust the particle position, making sure to cancel any speed to opposes the fixup.

	float LimitRadians = FMath::DegreesToRadians( AngleLimit );
	if( SwingRadians > LimitRadians )
	{
		FQuat FixupRotation( SwingAxis, LimitRadians - SwingRadians );
		FVector FixupPos = InBoneWS.GetLocation() + FixupRotation * CurrentOff;
		FVector FixupOffset = FixupPos - ParticlePos;
		if( ( FixupOffset | ParticleVel ) < 0.0f ) // dot-product check
			ParticleVel -= ParticleVel.ProjectOnTo( FixupOffset );
		ParticlePos = FixupPos;
		Swing = FQuat( SwingAxis, LimitRadians );
	}

Next I check the particle distance and pull it forward or push it back if it’s too-close or too far (this prevents it from getting “stuck” for extended periods of time when you move really fast).

	FVector ParticleOffset = ParticlePos - InBoneWS.GetLocation();
	float DistSq = ParticleOffset.SizeSquared();
	if( DistSq > 1e-5f ) // check to avoid a divide-by-zero edge-case
	{
		float MinDistSq = 0.666f * 0.666f * LocalTarget.SizeSquared();
		float MaxDistSq = 1.333f * 1.333f * LocalTarget.SizeSquared();
		if( DistSq < MinDistSq )
			ParticlePos = InBoneWS.GetLocation() + ParticleOffset * FMath::Sqrt( MinDistSq / DistSq );
		else if( DistSq > MaxDistSq )
			ParticlePos = InBoneWS.GetLocation() + ParticleOffset * FMath::Sqrt( MaxDistSq / DistSq );
	}

I determined with testing that about 1/3 further-out than normal is a limit that looks good in general (animation is full of fun magic numbers like this).

Finally, we compose the final result rotation, convert it from world-space back to component-space, and write it out:

	FQuat ResultWS = Swing * InBoneWS.GetRotation(); // note that FQuat * order is _NOT_ transposed
	FQuat ResultCS = ToWorld.GetRotation().Inverse() * ResultWS;
	OutBoneTransforms.Emplace( BoneIndex, FTransform( ResultCS, InBoneCS.GetLocation(), InBoneCS.GetScale3D() ) );
}

As always, this is just a base sample from which more complex AnimNodes can be forked.

Unity Code Listing

I’ll implement MonoBehavior for clarity, though you’d get better performance structuring this effect as an AnimationScriptPlayable job.

class TrailingBone
{
	public Vector3 localTarget;
	public float dampSeconds = 0.25f;
	public float angleLimit = 30.0f;

	Vector3 particlePos;
	Vector3 particleVel;

	/*
	A little hack to restore the my rest-pose every update.
	so there isn't a feedback-loop.  If you're downstream from an animation 
	that's over-writing the transform, you can skip this part.
	*/
	Quaternion jointRestPose;

	void Start()
	{
		particlePos = transform.TransformPoint( localTarget );
		jointRestPose = transform.localRotation;
	}

	void Update()
	{
		transform.localRotation = jointRestPose;

		// simulation
		var jointPos = transform.position;
		var particleTarget = transform.TransformPoint( localTarget );
		particlePos = Mathf.SmoothDamp( particlePos, particleTarget, ref particleVel, dampSeconds );
		
		// angle limit
		var targetOffset = particleTarget - jointPos;
		var currentOffset = particlePos - jointPos;
		var swing = Quaternion.FromToRotation( targetOffset, currentOffset );
		swing.ToAngleAxis( out var swingDegrees, out var swingAxis );
		if( swingDegrees > angleLimit )
		{
			var fixupRotation = Quaternion.AngleAxis( swingAxis, angleLimit - swingDegrees );
			var fixupPos = jointPos + fixupRotation * currentOffset;
			var fixupOffset = fixupPos - particlePos;
			if( Vector3.Dot( fixupOffset, particleVel ) < 0f )
				particleVel -= Vector3.Project( particleVel fixupOffset.normalized );
			particlePos = fixupPos;
			swing = QUaternion.AngleAxis( angleLimit, swingAxis );
		}

		// distance limit
		var particleOffset = particlePos - jointPos;
		var distSq = particleOffset.sqrMagnitude;
		if( distSq > Mathf.Epsilon )
		{
			var minDistSq = 0.666f * 0.666f * localTarget.sqrMagnitude;
			var maxDistSq = 1.333f * 1.333f * localTarget.sqrMagnitude;
			if( distSq < minDistSq )
				ParticlePos = jointPos + particleOffset * Mathf.Sqrt( minDistSq / distSq );
			else if( distSq > maxDistSq )
				ParticlePos = jointPos + particleOffset * Mathf.Sqrt( maxDistSq / distSq );
		}

		transform.rotation = swing * transform.rotation;
	}
}

Thanks for reading! ❤️

⚙️ Procedural Animation: Inverse Kinematics

Sun, 26 Feb 2023 00:00:00 +0000

Procedural animation is a priority on Tiny Starpilot. As a designer, I like how it it improves the responsiveness and “juiceness” of interactive characters. As a programmer, it’s my area of professional expertise. Finally, as a solo dev, it lets me create more “modular” assets which can be shared between characters with different skeletons.

The biggest tool in the procedural-animation toolkit is called Inverse Kinematics. With these algorithms, designers can specify “targets” (often called effectors) for the animation system, and joints are internally rotated to satisfy those goals.

An example of my modular-design: anim nodes automatically discover and coordinate with very little explicit setup.

Unreal has built-in IK, Control Rig, but I’ve chosen to write my own solvers both so that I can produce tighter results than their generic FABRIK solver, and to reduce the amount of boilerplate blueprint noodling.

A full study of IK would fill a college degree, so just to give a taste let’s break-down the simplest but most-common type of IK: two-bone solving. I use these to pose legs when generating locomotion, and to pose arms when aiming or grasping.

Given a “Base Pose” for a limb, compute the smallest shoulder/elbow rotation which places the point P3 onto the Target Effector.

What makes this case simple is that, unlike many other problems in kinematics, there is an analytical solution! It’s based on the Law of Cosines!

Given the length of your arm bones (A & B), and the offset from your shoulder to your hand (C), we can compute the angle you elbow makes (θ).

Applying this in code introduces two wrinkles:

How do we translate from the flat plane to 3D spatial coordinates?
How do we convert the angles into 3D orientations?

Let’s start writing code and I’ll give you those answers in context. Let’s start by declaring a helper-struct with all our inputs:

struct FTwoBoneSolver
{
	FTransform Root; // shoulder
	FTransform Mid;  // elbow
	FTransform End;  // hand

	void Solve( const FVector& Target ); // updates transforms in-place
};

FTransform is Unreal’s Position/Rotation/Scale structure.

A quick note at this point – all the joint transforms are in the same-coordinate space (in Animation Blueprints I use Component Space for everything). If you have relative transforms you’ll need to compose them, like so:

FTwoBoneSolver Solver;
// convert relative transforms to same coordinate space
Solver.Root = LocalShoulder;
Solver.Mid = LocalElbow * Solver.Root;
Solver.End = LocalHand * Solver.Mid;

...

// restore transform to relative local space
ShoulderTransform = Solver.Root;
ElbowTransform = Solver.Mid.GetRelativeTransform( Solver.Root );
HandTransform = Solver.End.GetRelativeTransform( Solver.Mid );

Note that Unreal FTransforms use transposed ( Child * Parent ) multiplication order (ugh)!

Now for the actual implementation, we start by saving the intial positions of the joints, and computing the bone lengths:

void FTwoBoneSolver::Solve( const FVector& Target )
{
	// input pose
	FVector InEndLoc = End->GetLocation();
	FVector InMidLoc = Mid->GetLocation();
	FVector InRootLoc = Root->GetLocation();
	float UpperLen = FVector::Dist( InRootLoc, InMidLoc );
	float LowerLen = FVector::Dist( InMidLoc, InEndLoc );
	float MaxLength = UpperLen + LowerLen - 1.0f;

Max Length is the maximum offset we can take the hand from the shoulder. This can’t be longer than the arm itself (unless you allow stretching), and I actually clamp it 1cm shorter to avoid some edge-cases which creep in when all three joints snap into a straight line, and you can’t determine their plane.

The next step is to recognize that the three points lie on a flat plane, and can be decomposed into two orthogonal “basis vectors” (ToEnd and the Pole Vector).

All our 2D geometry will be multiplied by vectors to become 3D coordinates.

	FVector ToEnd = ( InEndLoc - InRootLoc ).GetSafeNormal();
	FVector InPoleVec = FVector::VectorPlaneProject( InMidLoc - InRootLoc, ToEnd );
	InPoleVec.Normalize();

Next we compute the Swing rotation from ToEnd to ToTarget. This rotation is a Quaternion, which is just a technical way of encoding an Angle-Axis Rotation. Internally FindBetween() is using the cross-product to compute the axis of rotation and the dot-product to compute the angle.

	FVector TargetOffset = ( Target - InRootLoc ).GetClampedToMaxSize( MaxLength );
	float TargetDist = TargetOffset.Size(); // we'll want this later
	FVector ToTarget = TargetOffset / TargetDist;
	FQuat ToTargetSwing = FQuat::FindBetweenNormals( ToEnd, ToTarget );
	FVector OutPoleVec = ToTargetSwing * InPoleVec;

Note that ( Quaternion * X ) is a shorthand for Quaternion.RotateVector( X ).

Now we have all the inputs to the Law of Cosines, as well as the vectors to convert them from 2D to 3D.

A = TargetDist
B = UpperLen
C = LowerLen

Rearrange the terms to solve for θ, the angle the Upper Arm bone will make with the ToTarget direction:

When we plug this in we calculate the denominator first, so we can check for a divide-by-zero edge case:

	float Denom = 2.0f * UpperLen * TargetDist;
	float CosAngle = 0.0f;
	if( Denom > KINDA_SMALL_NUMBER )
		CosAngle = ( TargetDist*TargetDist + UpperLen*UpperLen - LowerLen*LowerLen ) / Denom;
	float Angle = FMath::Acos( CosAngle );

From here it’s just some trigonometry to determine the offsets along the ToTargetDir and OutPoleVec basis directions (recall that cosθ and sinθ are the horizontal and vertical components of a 2D unit direction vector):

	float PoleDist = UpperLen * FMath::Sin( Angle );
	float TargetDist = UpperLen * CosAngle;
	FVector OutEndLoc = InRootLoc + TargetOff;
	FVector OutMidLoc = InRootLoc + TargetDist * ToTargetDir + PoleDist * OutPoleVec;

So now we have the new end and mid positions (the shoulder doesn’t move). To convert these to rotations we use the same FindBetween() functions, comparing the old and new directions. Do note that the MidSwing is downstream from the RootSwing, and therefore needs to combine both rotations.

	FVector ToMidIn = InMidLoc - InRootLoc;
	FVector ToMidOut = OutMidLoc - InRootLoc;
	FQuat RootSwing = FQuat::FindBetween( ToMidIn, ToMidOut );
	FVector InEndLoc_WithRootSwing = InRootLoc + RootSwing * ( InEndLoc - InRootLoc );
	FVector ToEndIn = InEndLoc_WithRootSwing - OutMidLoc;
	FVector ToEndOut = OutEndLoc - OutMidLoc;
	FQuat MidSwing = FQuat::FindBetween( ToEndIn, ToEndOut ) * RootSwing;

Unlike FTransforms, FQuats use the textbook ( Parent * Child ) multiplication order (YOLO)

And the last thing we do is write the results:

	Root->SetRotation( RootSwing * Root->GetRotation() );
	Mid->SetLocation( OutMidLoc );
	Mid->SetRotation( MidSwing * Mid->GetRotation() );
	End->SetLocation( OutEndLoc );
	End->SetRotation( MidSwing * End->GetRotation() );
}

Before we wrap, I’ll show you a snippet of this code in use to clarify what I meant up top by writing custom code to get “tighter results” with less “boilerplate.” Here’s the actual Animation Evaluation code for the arm-aiming modifier. The solver gets the arm in basically the right pose, and then we apply a small FindBetween() fixup rotation to swing the whole arm so the forearm direction matches your shooting direction.

Pew, pew, pew!

void FAnimNode_ArmAim::EvaluateSkeletalControl_AnyThread( FComponentSpacePoseContext& Output, TArray<FBoneTransform>&  OutBoneTransforms )
{
	// initialize with the base pose from the previous anim node
	FTwoBoneSolver Solver;
	Solver.Root = Output.Pose.GetComponentSpaceTransform( ShoulderIdx );
	Solver.Mid = Output.Pose.GetComponentSpaceTransform( ElbowIdx );
	Solver.End = Output.Pose.GetComponentSpaceTransform( WristIdx );

	// extent arm almost-full (99%) towards the target
	float Length = 0.99f * (
		FVector::Dist( Solver.Root.GetLocation(), Solver.Mid.GetLocation() ) +
		FVector::Dist( Solver.Mid.GetLocation(), Solver.End.GetLocation() )
	);
	Solver.Solve( Solver.Root.GetLocation() + AimingDir * Length );
	
	// align the elbow->hand direction to the aiming direction
	FVector ForearmDir = Solver.End.GetLocation() - Solver.Mid.GetLocation();
	FQuat Fixup = FQuat::FindBetween( ForearmDir, AimingDir );
	Solver.Mid.SetLocation( Solver.Root.GetLocation() + Fixup * ( Solver.Mid.GetLocation() - Solver.Root.GetLocation() ) );
	Solver.End.SetLocation( Solver.Root.GetLocation() + Fixup * ( Solver.End.GetLocation() - Solver.Root.GetLocation() ) );
	Solver.Root.SetRotation( Fixup * Solver.Root.GetRotation() );
	Solver.Mid.SetRotation( Fixup * Solver.Mid.GetRotation() );
	Solver.End.SetRotation( Fixup * Solver.End.GetRotation() );

	OutBoneTransforms.SetNumUninitialized(3);
	OutBoneTransforms[0] = FBoneTransform( ShoulderIdx, Solver.Root );
	OutBoneTransforms[1] = FBoneTransform( ElbowIdx, Solver.Mid );
	OutBoneTransforms[2] = FBoneTransform( WristIdx, Solver.End );
}

Fun fact: Unreal can evaluate FAnimNodes on background threads in parallel, so we can actually go a bit hog-wild with the math.

UPDATE: Here’s a complete listing of the same code for the Unity Game Engine in C#:

using UnityEngine;

public struct JointPose {
	public Vector3 position;
	public Quaternion rotation;
}

public struct TwoBoneIK {
	public JointPose root;
	public JointPose mid;
	public JointPose end;

	public void Solve( Vector3 InTargetPos ) {
		var InEndLoc = end.position;
		var InMidLoc = mid.position;
		var InRootLoc = root.position;
		var UpperLen = Vector3.Distance( InRootLoc, InMidLoc );
		var LowerLen = Vector3.Distance( InMidLoc, InEndLoc );
		var MaxLength = UpperLen + LowerLen - 0.01f;

		var ToEnd = ( InEndLoc - InRootLoc ).normalized;
		var InPoleVec = Vector3.ProjectOnPlane( InMidLoc - InRootLoc, ToEnd ).normalized;

		var ToTargetOffset = ( InTargetPos - InRootLoc );
		if( ToTargetOffset.sqrMagnitude > MaxLength * MaxLength )
			ToTargetOffset = ToTargetOffset.normalized * MaxLength;
		var ToTargetDist = ToTargetOffset.magnitude;
		var ToTarget = ToTargetOffset / ToTargetDist;
		
		var ToTargetSwing = Quaternion.FromToRotation( ToEnd, ToTarget );
		var OutPoleVec = ToTargetSwing * InPoleVec;

		var Denom = 2.0f * UpperLen * ToTargetDist;
		var CosAngle = 0.0f;
		if( Denom > Mathf.Epsilon )
			CosAngle = ( ToTargetDist*ToTargetDist + UpperLen*UpperLen - LowerLen*LowerLen ) / Denom;

		var Angle = Mathf.Acos( CosAngle );
		var PoleDist = UpperLen * Mathf.Sin( Angle );
		var EffDist = UpperLen * CosAngle;

		var OutEndLoc = InRootLoc + ToTargetOffset;
		var OutMidLoc = InRootLoc + EffDist * ToTarget + PoleDist * OutPoleVec;

		var InToMid = InMidLoc - InRootLoc;
		var OutToMid = OutMidLoc - InRootLoc;
		var RootSwing = Quaternion.FromToRotation( InToMid, OutToMid );
		var InEndLoc_WithRootSwing = InRootLoc + RootSwing * ( InEndLoc - InRootLoc );
		var ToInEnd = InEndLoc_WithRootSwing - OutMidLoc;
		var ToOutEnd = OutEndLoc - OutMidLoc;
		var MidSwing = Quaternion.FromToRotation( ToInEnd, ToOutEnd ) * RootSwing;

		root.rotation = RootSwing * root.rotation;
		mid.position = OutMidLoc;
		mid.rotation = MidSwing * mid.rotation;
		end.position = OutEndLoc;
		end.rotation = MidSwing * end.rotation;
	}
}

Thanks for reading! ❤️

⚙️ Tech Breakdown: Collision Sliding

Mon, 13 Feb 2023 00:00:00 +0000

I’ve begun consolidating my various demos and prototypes into a single Unreal Engine project.

A big feature to resolve with this port is how collision is handled. Let’s discuss collision handling in general, and how to start implementing it in Unreal in particular. This will be a code-heavy post (C++).

The gang is looking pretty Unreal.

In my Unity-based Character-Control demo I handled all collision using the Rigidbody component. This has the advantage of being quick & easy for early prototyping, because all the collision-handling and interactions with other moving-objects mostly “just works.”

It’s not physics if you don’t have boxes bouncing around.

The drawback is that rigidbody collision responses are generic. Unless you’re making a physics-sandbox game, it’s hard to dial-in tight controls. So for the port I want to start writing my own collision handling.

The full implementation of a colliding character controller is beyond the scope for one blog post, but we can at least start with a simple foundation: sliding.

We sweep forward to an impact, and decompose the remainder into the “tangent” which lies along the surface.

To implement this in Unreal, I started by declaring a helper object. This technique keeps code modular while still portable to different parts of the game (we can embed them in Actors, Components, Anim Nodes, Blueprint Library Functions, etc), as well as being easy to copy-paste between projects without bringing a lot of other cruft with them.

class FSlide
{
public:

	// out start position
	FVector Position = FVector( 0.0f );

	// the desired offset vector
	FVector Remainder = FVector( 0.0f );

	// the shape to slide
	FCollisionShape Shape = FCollisionShape::MakeSphere( 20.0f );

	// the rotation of the shape
	FQuat Rotation = FQuat::Identity;

	// parameters used by unreal's collision-query system
	FCollisionQueryParams QueryParams = FCollisionQueryParams::DefaultQueryParam;
	ECollisionChannel Channel = ECC_Pawn;

	// Advance Position/Remainder to the next impact
	bool TryStep( const UWorld* World, FHitResult& Hit );

private:

	// Will be relevant later ;)
	FVector PrevNormal = FVector( 0.0f );
};

Before diving into the details of this function, let’s write a quick little test actor. I’ve omitted a complete code-listing for brevity, but everything happens in Tick():

/*virtual*/ void ATestSlidePawn::Tick( float DeltaTime ) /*override*/
{
	Super::Tick( DeltaTime );

	// Only update when player has control
	if( !Controller )
	{
		return;
	}

	UWorld* World = GetWorld();

	// gamepad stick inputs
	const FVector2D CamInput ( GetInputAxisValue( NAME_CamX ), GetInputAxisValue( NAME_CamY ) );
	const FVector2D MoveInput ( GetInputAxisValue( NAME_MoveX), GetInputAxisValue( NAME_MoveY ) );

	// use the right-stick to rotate the camera
	FRotator ControlEuler = Controller->GetControlRotation();
	ControlEuler.Yaw += CamInput.X * 120.0f * DeltaTime;
	ControlEuler.Yaw = FRotator::NormalizeAxis( ControlEuler.Yaw );
	ControlEuler.Pitch -= CamInput.Y * 100.0f * DeltaTime;
	ControlEuler.Pitch = FMath::Clamp( ControlEuler.Pitch, -70.0f, 70.0f );
	Controller->SetControlRotation( ControlEuler );

	// compute a desired velocity by mapping the left stick relative to the camera
	const FQuat CamRot = ControlEuler.Quaternion();
	const FVector Vel = CamRot *  FVector( 500.0f * MoveInput.Y, 500.0f * MoveInput.X, 0 );

	// perform the slide
	FSlide Slide;
	Slide.Position = GetActorLocation();
	Slide.Remainder = Vel * DeltaTime;
	Slide.QueryParams = FCollisionQueryParams( NAME_Test, false, this );

	// cap out at three impacts
	FHitResult Hit;
	for( int It = 0; It < 3 && Slide.TryStep( World, Hit ); ++It )
	{
		if( Hit.bStartPenetrating )
		{
			// Stuck! Draw a red warning sphere
			DrawDebugSphere( World, Slide.Position, Slide.Shape.GetSphereRadius() + 5.0f, 16, FColor::Red );
			break;
		}
		else
		{
			// Impact! Draw the surface normal
			DrawDebugDirectionalArrow( World, Hit.ImpactPoint, Hit.ImpactPoint + 65.0f * Hit.Normal, 10.0f, FColor::Green, false, 0.5f );
		}
	}

	// update the actor and camera positions
	SetActorLocation( Slide.Position );
	CameraComp->SetWorldLocationAndRotation( Slide.Position - CamRot * FVector( 300, 0, 0 ), CamRot );
}

Here’s a naive first-stab at the FSlide::TryStep() function:

bool FSlide::TryStep( const UWorld* World, FHitResult& Hit )
{
	// early out?
	if( Remainder.IsNearlyZero( KINDA_SMALL_NUMBER ) )
	{
		return false;
	}

	// sweep forward
	const bool bHit = World->SweepSingleByChannel( Hit, Position, Position + Remainder, Rotation, Channel, Shape, QueryParams );
	if( !bHit )
	{
		Position = Hit.TraceEnd;
		Remainder = FVector( 0.f );
		return false;
	}

	// stuck?
	if( Hit.bStartPenetrating )
	{
		Remainder = FVector( 0.0f );
		return true;
	}

	// update position and remainder
	Position = Hit.Location;
	Remainder *= ( 1.f - Hit.Time );
	Remainder = FVector::VectorPlaneProject( Remainder, Hit.Normal );

	return true;
}

Seems to do everything in the picture: we use Unreals’ SweepSingleByChannel method to cast for an impact in the scene, and then either fully or partially advance based on whether we get a hit. Let’s take it for a spin:

Derp.

As soon we we touch collision, we get stuck. Wat?

The problem is precision. Positions in Unreal Engine 4 are represented using single-precision floating-point numbers, which on a good-day only get you about 6-8 digits (or “significant figures”). Because of these tiny errors, if we get too-close to a surface we actually start overlapping it a little bit, which causes SweepSingleByChannel to report the bStartPenetrating condition.

Unreal Engine 5 eases this situation somewhat by using double-precision floats, but the issue persists.

Let’s try “pulling back” slightly from surfaces when updating position when we advance to create a gap for the error.

FVector PullBackMove( const FVector& Move )
{
	// magic number I found elsewhere in the Unreal Engine source code
	const float Epsilon = 0.125f; 
	const float Dist = Move.Size();
	return Dist > Epsilon ? Move * ( ( Dist - Epsilon ) / Dist ) : FVector( 0.0f );
}

Then we change how the position is updated.

// new position update
//Position = Hit.Location;
Position = Hit.TraceStart + PullBackMove( Hit.Location - Hit.TraceStart );

This actually does a pretty good job. We’re not getting stuck on simple surfaces, but we can still get stuck rubbing against complex mesh-geo collision.

Better.

The issue is that the pull-back doesn’t work at glancing angles. And there’s also a broader issue: sometimes you can’t avoid little overlaps, e.g. if you have moving world geometry that encroaches on your shape independently of precision. What we really need is to detect and depenetrate from initial overlaps before sweeping.

The FHitResult contains minimum-separating-vector information in its Normal and PenetrationDepth fields when it raises the bStartPenetrating condition, but it’s actually kind of useless because it only reports a single result, whereas shapes can logically have multiple initial overlaps – especially in narrow corners.

A pathological, but not uncommon, case – there are multiple initial overlaps, and their separation-vectors all oppose each other.

Unreal provides the SweepMulti variant specifically to gather infomation to solve this case.

When bStartPenetrating is raised, the results-array contains all the initial overlaps.
When the sweep succeeds it finds exactly-zero-or-one blocking impact (at the end of the results), but also reports all the other overlaps along the path, which can be used to detect trigger-volumes with the same queries used for collision-responses.

To use this, first we’ll want a helper function to “inflate” a shape, because when we depenetrate we want to use a shape with a thin “skin” which will leave an airgap for the resweep.

FCollisionShape InflateShape( const FCollisionShape& Shape, float Amount )
{
	// helper function to add a "skin" to a shape
	switch( Shape.ShapeType )
	{
	case ECollisionShape::Capsule: 
		return FCollisionShape::MakeCapsule( Shape.GetCapsuleRadius() + Amount, Shape.GetCapsuleHalfHeight() + Amount );
	case ECollisionShape::Sphere:  
		return FCollisionShape::MakeSphere( Shape.GetSphereRadius() + Amount );
	case ECollisionShape::Box:     
		return FCollisionShape::MakeBox( Shape.GetBox() + Amount );
	default: 
		return Shape;
	}
}

With that in hand, here’s a wrapper for SweepMultiByChannel which detects initial penetration, wiggles us free, and then resweeps if needed.

bool DepenAndSweep( const UWorld* World, FHitResult& Hit, const FVector& Start, const FVector& Delta, const FQuat& Rot, ECollisionChannel Channel, const FCollisionShape& Shape, const FCollisionQueryParams& Params )
{
	// "Scratchpad" for result list, to avoid reallocating results for each sweep
	// (NOTE: makes this function not threadsafe!)
	static TArray< FHitResult > Hits;
	Hits.Reset();

	// sweep with inflated skin
	FCollisionShape Inflated = InflateShape( Shape, 0.25f );
	const bool bSkinHit = World->SweepMultiByChannel( Hits, Start, Start + Delta, Rot, Channel, Inflated, Params );

	// no hits, even with skin?
	if( !bSkinHit )
	{
		Hit.Init();
		Hit.TraceStart = Start;
		Hit.TraceEnd = Start + Delta;
		Hit.Location = Start + Delta;
		Hit.Time = 1.0f;
		return false;
	}

	// have initial overlaps?
	int NumInitialOverlaps = 0;
	for( const FHitResult& It : Hits )
	{
		if( It.bStartPenetrating )
		{
			++NumInitialOverlaps;
		}
	}
	if( NumInitialOverlaps == 0 )
	{
		Hit = Hits.Last();
		return bSkinHit;
	}
	
	// iteratively resolve penetration
	FVector Fixup( 0.0f );
	for( int Iter = 0; Iter < 16; ++Iter )
	{
		float ErrorSum = 0.0f;
		for( const FHitResult& It : Hits )
		{
			if( It.bStartPenetrating )
			{
				// take the dot-product of the Fixup and the Normal to determine how much
				// of the penetration has already been taken care of.
				const float Error = FMath::Max( 0.0f, ( It.PenetrationDepth + 0.125f ) - ( Fixup | It.Normal ) );
				ErrorSum += Error;
				Fixup += Error * It.Normal;
			}
		}
		// If we've found a solution, stop iterating.
		if( ErrorSum < KINDA_SMALL_NUMBER )
		{
			break;
		}
	}

	// resweep from new start
	return World->SweepSingleByChannel( Hit, Start + Fixup, Start + Fixup + Delta, Rot, Channel, Shape, Params );
}

We use numeric iteration to resolve the fixup because there’s no cheap analytical solution. Having nested inner-loops like this might smell like bad perf, but the calculations are so cheap that a modern CPU won’t even break a sweat.

UPDATE: @ctangora assures me that this resolver is actually equivalent to a Gauss-Seidel Method, and she’s much smarter than me, so I trust her. ❤️

Success!

It’s technically still possible to get stuck, even with this method, but it’s highly unlikely and content-dependent, so those bugs can be fixed on a case-by-case basis. On Solar Ash, e.g., we’d see Rei get stuck about once every two hours or so during QA, so we just added a little white-noise to stuck positions and they’d pop-out within a few frames.

Between the articulated moving platforms, detailed mesh collision, and changing gravity you can imagine how wild the collision issues were.

And that’s about it for sliding. Note that this code is not meant to be used verbatim, but as a starting point. As your controls are developed, you’ll add different responses to different surface-types and movement-modes. E.g. a grounded character will treat the floor, walls, and ceilings differently.

One last cool trick before I wrap this up. As implemented so far, you’ll see some jitter when you try and slide between surfaces that meet at an acute angle. The solution is to recognize that in this case we actually want to slide along the seam in between them. Our helper stores the PrevNormal each step to compute this. Here’s a full listing along with the subtituted call to DepenAndSweep:

bool FSlide::TryStep( const UWorld* World, FHitResult& Hit )
{
	// early out?
	if( Remainder.IsNearlyZero( KINDA_SMALL_NUMBER ) )
	{
		return false;
	}

	// sweep forward
	const bool bHit = DepenAndSweep( World, Hit, Position, Remainder, Rotation, Channel, Shape, QueryParams );
	if( !bHit )
	{
		Position = Hit.TraceEnd;
		Remainder = FVector( 0.f );
		return false;
	}

	// stuck?
	if( Hit.bStartPenetrating )
	{
		return true;
	}

	// update position
	Position = Hit.TraceStart + PullBackMove( Hit.Location - Hit.TraceStart );

	// remainder is either projected onto the impact plane, or the seam between two acute impact-planes
	// dot-product to check the angle
	// cross product to calcuate the seam
	const FVector Seam = ( PrevNormal | Hit.Normal ) < KINDA_SMALL_NUMBER ? ( PrevNormal ^ Hit.Normal ) : FVector( 0.0f ) ;
	Remainder *= ( 1.f - Hit.Time );
	Remainder = Seam.IsNearlyZero( 0.1f ) ? 
		FVector::VectorPlaneProject( Remainder, Hit.Normal ) : 
		Remainder.ProjectOnTo( Seam ) ;
	PrevNormal = Hit.Normal;

	return true;
}

Until next time, happy sliding!

🖌️ Project Update: Playable Pilot Concepts

Fri, 10 Feb 2023 00:00:00 +0000

I’ve planned for three playable pilots since my earliest prototype.

The design goal is straightforward: approaching the same mission with different pilots who handle differently increases replay variety. It also acts as a difficulty handicap for casual or hardcore players to opt-into by choosing particular easy or challenging pilots for particular maps.

This week I was feeling ambitious and decided to flex my illustration muscles, and made a nice set of pilot-portrait key-art, as well as sketch concepts for their fighter craft. I really want to push the 80s mecha-anime space-opera creative direction.

My original fat-pixel lineup.

These are all still early concepts, and might change a lot. For their fighter craft, I tried to limit myself to things which can be modelled in the low-poly style, so I kept them sketchier. The one thing I didn’t do was choose names. It’s too hard!

The Rebel Ace

“Wait, that was supposed to be a challenge?”

The Ace is the face for the game: a cocky protagonist with a fiery sense of justice who’s driven to protect civilians. Her primary flaw is her pride. I think of her as a mix of Kei, Judau Ashta, and Lufy.

If I were this cute, I’d be vain too.

Her search-and-rescue fighter craft is the balanced gameplay choice, and has a good mix of firepower and evasiveness. It’s special feature is the ability to transform between mecha and jet forms.

The Imperial Commander

“The only real challenge to power is power itself.”

The Commander is the heel: an cold-hearted but charismatic authoritarian prone to pontificating on his political ideals. He’s the ace’s natural rival. I basically started with Char Aznable, and kept pressing the big button labelled “Sephiroth” until his design screamed “daddy!”

Classic callback.

His attack mecha is the tank choice, with high armor and ammo-payload in exchange for limited maneuverability. It basically turns the game into a shooting gallery like Omega Boost. His special is an ultra-heavy charge-attack cannon.

The Space Pirate

“Who’s content to live in a fishbowl once you’ve ridden the sea of stars?”

The Pirate is the wildcard: a smuggler who delights in making her own way in the world through her own skill and wits, outside the law. But behind her hedonistic facade, lies a principled sense of honor. A total throwback to Captain Harlock.

This was actually the first model I made for this game.

Her fighter is the glass cannon choice, offering high-speed and dodge-maneuvers in exchange for lighter sniper-style firepower and no armor. More risk for more reward: her special increases the amount of loot you receive.

⚙️ Math Breakdown: Anime Homing Missiles

Tue, 31 Jan 2023 00:00:00 +0000

I designed and prototyped the missile attack! The math was clever and I want to show-off!

Let’s talk about cubic bezier curves, perlin noise, and rotation minimizing frames.

Doing Ichirō Itano proud.

I’ll keep this article a little lighter on code. I really want to focus instead on the geometry. Many people are intimidated by math, but keep in mind that you don’t need to understand everything to use it.

Even if this is retreading topics you’ve already mastered, hopefully my own solution shows you some new ways you can combine these techniques creatively.

Iterative vs. Closed-Form Motion

Broadly speaking, there are two flavors of movement code. Iterative Code updates an object position incrementally one frame at a time, in a process college-professors call integration. A common example of this Euler’s Method where we compute the velocity of the object and then nudge the position in that direction over a timestep:

void Update( float DeltaTime ) {
	Vector3 Velocity = CalculateVelocity();
	Vector3 Position = GetPosition();
	SetPosition( Position + DeltaTime * Velocity );
}

Delta is just ‘math speak’ for “change in” as in “the change in time this Update()”.

This is the natural way to write character control, in which the player input varies every frame, or complex physics simulations where there is no known realtime analytical solution.

Alternatively, if you know the whole motion ahead of time, you can use Closed-Form Code where you plot the whole path from initial conditions (math heads call it a parametric curve), and sample the current time. A good example of this is the well-known Cubic Bezier Curve:

Vector3 CalcBezierPos( Vector3 P0, Vector3 P1, Vector3 P2, Vector3 P3, float t ) {
	float t_ = 1 - t;
	return
		(t_ * t_ * t_)    * P1 + 
		(3 * t_ * t_ * t) * P2 +
		(3 * t_ * t * t)  * P3 + 
		(t * t * t)       * P4 ;
}

If you’ve ever used any vector graphics tool you probably recognize this. Beziers are cubic polynomials which is a fancy way of saying the simplest path with four degrees of freedom: the endpoints P0 and P3 and the “control points” P1 and P2 which affect the orientation and curvature.

The input t is called the input parameter and is a ratio on the range 0-1. So e.g. t=0.333 is about a third of the way through. To move a point, we simply take the elapsed time since the start of the motion, divided by the duration.

float StartTime;
float Duration;

void Update() {
	float CurrentTime = Time.time;
	float Elapsed = CurrentTime - StartTime;
	if( Elapsed >= Duration )
		SetPosition( P3 ); // we're at the end
	else
		SetPosition( CalcBezierPos( P0, P1, P2, P3, Elapsed / Duration ) );
}

In addition to the position, we can also use the bezier parameters to calculate the derivative at t, which is the rate of change. This vector is useful because we say it’s tangent to the curve, i.e. it points in the direction of the motion. To convert this to speed, divide by the duration.

Vector3 CalcBezierDeriv( Vector3 P0, Vector3 P1, Vector3 P2, Vector3 P3, float t ) {
	float t_ = 1 - t;
	return  (
		( 3 * t_ * t_ ) * ( P1 - P0 ) + 
		( 6 * t_ * t ) * ( P2 - P1 ) + 
		( 3 * t * t ) * ( P3 - P2 ) ;
}

float Velocity = CalcBezierDeriv( P0, P1, P2, P3, Elapsed / Duration ) / Duration;

Speed = Meters Per Second = ( Meters Per T ) / ( Seconds Per T ) = Deriv / Duration

Simulating Homing Missiles

Because I know where the homing missile path starts (the launcher muzzle), and where it ends (the painted target), I chose to use a bezier curve as the base for the homing missile path.

P1 is placed in front of the shooter, and P2 is projected out from the target surface.

Using a closed form kept things simple, because I didn’t have to solve a complicated “simulation” that hits the right point, and I can fine-tune the exact time between when you fire and when it hits, which is more intuitive than second-order physics units.

This was a perfectly cromulent effect, if a bit bland. We can do better.

Adding Noise

With a solid foundation, it’s time to start adding juice. Missile storm attacks in anime take erratic paths with more dynamism. We can simulate this by adding noise.

A common go-to for FX artists is Perlin Noise, a kind-of pseudorandom oscillation – it’s erratic, but also smooth. The code’s a little too long to post here, but it’s not hard to find samples online.

Search for “Simplex Noise” (the name of a common optimized variant).

An obvious problem here is that I need the offset to be zero at endpoints, so the missile lines up with the muzzle and the target. I achieved this by multiplying it by an envelope which is zero at the ends and one in the middle.

How do we turn a noise function into a 3D curve-deformed offset? We compute two independent noise values and use them as the X and Y components of an offset vector transformed by a rotation-frame that’s aligned to the bezier’s derivative (what a mouthful!).

Vector3 LocalOffset;
float NoiseFreq = 2f; // tuning value for wiggle frequency
float NoiseAmp = 8;   // tuning value for wiggle size
float Envelope = 1 - (1 - 2 * t) * (1 - 2 * t);
LocalOffset.x = NoiseAmp * Envelope * Noise( NoiseSeedX, NoiseFreq * Elapsed );
LocalOffset.y = NoiseAmp * Envelope * Noise( NoiseSeedY, NoiseFreq * Elapsed );
LocalOffset.z = 0;
Quaternion Frame = Quaternion.LookRotation( CalcBezierDeriv( P0, P1, P2, P3, t ) );
SetPosition( CalcBezierPos( P0, P1, P2, P3, t ) + Frame * LocalOffset );

TL;DR We’re wiggling along the red and green arrows.

This almost worked, but I saw some glitches. Vertically-Locked rotation frames calculated this way are aligned to the derivative, but twist a lot – especially when the path is vertical. Instead, we want so-called Rotation Minimizing Frames which have no instantaneous twist, just minimal swings between directions.

(A) Vertically-Locked Frames (B) Rotation Minimizing Frames

The method of computing minimizing frames is mathematically complex in general, but luckily a paper was published in 2006 which described a shockingly simple method for “nudging” a frame forward without twisting called the Double Reflection Method. We don’t need to understand the derivation, we can just know that it’s cheap and it works.

Shoutout to Jasper St. Pierre for showing me this mathy-ass math.

Quaternion Frame; // Initialize to Quaternion.LookDirection( P1 - P0 );

void UpdateFrame( float t ) {
	// starting "normal" and "tangent"
	var n0 = Frame * Vector3.up;
	var t0 = Frame * Vector3.forward;

	// target "tangent"
	var t1 = CalcBezierDeriv( P0, P1, P2, P3, t ).normalized;

	// first reflection
	var v1 = CalcBezierPos( P0, P1, P2, P3, t ) - GetPosition(); 
	var c1 = v1.sqrMagnitude;
	var n0_l = n0 - (2 / c1) * Vector3.Dot(v1, n0) * v1;
	var t0_l = t0 - (2 / c1) * Vector3.Dot(v1, t0) * v1;

	// second reflection
	var v2 = t1 - t0_l;
	var c2 = v2.sqrMagnitude;
	var n1 = n0_l - (2 / c2) * Vector3.Dot(v2, n0_l) * v2;

	// build rotation with target normal for "up"
	Frame = Quaternion.LookRotation( t1, n1 );
}

Et Voilà!

Thanks for reading! I promise I’ll do an art post next to give everyone a break from code 🙏

⚙️ Tech Breakdown: Bullet Hell

Tue, 17 Jan 2023 00:00:00 +0000

I’m adding an Itano-style Missile Circus Attack right now, so let’s talk about the principles of a high-performance bullet-hell system.

Four thousand bullets ought to be enough for anyone.

Computers are fast. Your Nintendo Switch has four 1.02 GHz ARM Cortex-A57 CPU cores, each one 500x more powerful than the computer which sent us to the moon. So why is it slow to spawn a few thousand Bullet Objects?

It boils down to two issues: memory and memory (again).

These samples are for Unity, but the principles are engine-agnostic. For instance, I also applied them with the Unreal Game Engine in the development of Solar Ash. That translation is left as an exercise to the reader.

Memory Problem 1: Allocation and Object Pooling

Normally we add objects to the scene with Object.Instantiate(), which allocates managed memory for C# components as well as several handles to C++ native engine resources (meshes, transforms, etc). On removal, Object.Destroy() unregisters the resources, and releases its memory reference so that the garbage collector can come around later to return the RAM back to the system.

When this overhead is amortized over the lifetime of the object it doesn’t matter, but bullets are rapidly spawned and short lived, so we’re continuously creating them and tearing them down, which is a drag on performance.

An alternative is to pre-allocate a pool of hidden bullets. To spawn a bullet, we just grab an unused one, toggle its visibility, and put it in the right place. To remove, we simply hide it and put it back in the pool to be recycled.

I’ve implemented this several ways over the years, but here’s a pithy version that pleases me:

using UnityEngine;

public class PooledObject : MonoBehaviour
{
	/*
	For the prefab, next points at the head of the freelist.
	For inactive instances, next points at the next inactive instance.
	For active instances, next points back at the source prefab.
	*/
	[System.NonSerialized] PooledObject next;

	public static T Create<T>(T prefab, Vector3 pos, Quaternion rot) where T : PooledObject 
	{
		T result = null;
		if (prefab.next != null)
		{
			/*
			We have a free instance ready to recycle.
			*/
			result = (T) prefab.next;
			prefab.next = result.next;
			result.transform.SetPositionAndRotation(pos, rot);
		}
		else
		{
			/*
			There are no free instances, lets allocate a new one.
			*/
			result = Instantiate<T>(prefab, pos, rot);
			result.gameObject.hideFlags = HideFlags.HideInHierarchy;
		}
		result.next = prefab;
		result.gameObject.SetActive(true);
		return result;
	}

	public void Release()
	{
		if (next == null)
		{
			/*
			This instance wasn't made with Create(), so let's just destroy it.
			*/
			Destroy(gameObject);
		}
		else
		{
			/*
			Retrieve the prefab we were cloned from and add ourselves to its
			free list.
			*/
			var prefab = next;
			gameObject.SetActive(false);
			next = prefab.next;
			prefab.next = this;
		}
	}
}

You may be asking: I said there’d be a pool. Where is it? Shouldn’t there be a List<T> or something somewhere?

It’s actually all there in the next reference. The pool is a Linked List where each instance points at the next item in a list, like a chain. A picture is worth 1,000 words:

The prefab points at the first free bullet, and each free bullet points to the next free bullet. Active bullets point back at the prefab so they know where to go when they’re recycled.

All from one field.

With this method we can create and release bullets with a just few cheap assignments. Once the pool size stabilizes we no longer have to allocate or deallocate any memory for as long as the scene continues to run. As a nice bonus, we naturally get separate freelists for every type of prefab that’s pooled.

Challenge question: can you implement a function to free-up unused resourced by Destroy()ing the freelist?

Memory Problem 2: Context-Switching and Data-Oriented Design

Pooling eliminates the cost of creating and destroying bullets, but our profiler still shows a pretty siginifcant cost Update()ing them. In particular are two slow operations:

UnityEngine.Physics.Raycast checking for impact collisions
UnityEngine.Transform.position= updating the scene position

If you dig even deeper (e.g. using a hardware profiler like VTune), you’ll see that both of these methods spend most of their time fetching data. This is called a pipeline stall.

I won’t go too deep into this topic, since whole books have been written about it, but basically while the CPU is very fast, memory is very slow. If we can’t keep the CPU fed with data, it will spend most it’s time waiting around.

What follows is a simplified explanation that glosses over a lot, but is good-enough-for-game-programmers.

Memory is divided into two parts: a large pool of DRAM which is attached to the CPU over a bus, and a small cache of fast SRAM which is on the CPU die. When the CPU attempts to accesses uncached memory it’s called cache miss, which triggers a system interrupt to copy the data from main memory to the cache. Since the cache is limited, stale memory in the cache is retired-back to main memory to make room. Cache hits are fast, but misses are slow – dozens or even hundreds of cycles.

E.g. you could calculate several trig functions or multiply several matrices together during one single cache-miss.

During the bullet update, four distinct chunks of memory are accessed:

The executable machine code, or instructions, which are placed in a dedicated ICACHE.
The gameplay script data, stored in C# objects.
The PhysX spatial collision data, stored on the C++ heap.
The transform hierarchy data, stored natively in the engine.

ICACHE misses happen when code is interleaved. So for instance, code like this:

Foo(x);
Bar(x);
Baz(x);
Foo(y);
Bar(y);
Baz(y);
Foo(z);
Bar(z);
Baz(z);

… won’t be expected to perform as well (subject to complexity and compiler inlining) as sorted code like this:

Foo(x);
Foo(y);
Foo(z);
Bar(x);
Bar(y);
Bar(z);
Baz(x);
Baz(y);
Baz(z);

… because it’s busting the cache and reloading instructions on every other line, as opposed to reusing code while it’s still hot.

For bullets, this suggests first changing from every bullet having it’s own Update() interleaved with other scripts to a single BulletSystem which updates them all at once, e.g.

class BulletSystem
{
	Bullet[] bullets;
	int bulletCount;

	void Update()
	{
		float dt = Time.deltaTime;
		for (int it = 0; it < bulletCount; ++it)
		{
			var distance = bullets[it].speed * dt;
			if (Physics.Raycast(bullets[it].pos, bullets[it].dir, out var hit, dist))
			{
				// hit something
				bullet.Release();
				HandleBulletImpact(hit);
			}
			else
			{
				// advancing
				bullets[it].pos += distance * bullets[it].dir;
				bullets[it].transform.position = bullets[it].pos;
			}
		}
	}
}

Now all the position-calculations are performed together and the instructions only need to be fetched once. Furthermore, if bullets is an array of C# structs, as opposed to classes, then the state data will inline without indirection, further minimizing the cache miss rate (since nearby data is batched in pages and lines that are cached in chunks).

Depending on your access pattern, you can go even further – refactoring the bullets array into several parallel arrays for different fields like position and velocity. This is called the “Array of Structures to Structure of Arrays” transformation. Many programmers are so enamored of this technique that it can become something of a cargo cult.

But we still have the calls into Physics.Raycast and Transform.position= busting the cache accessing a bunch of data that’s not local to the UpdateBullets() script. Here’s an illustration:

Fig.1 All the data in main memory, attached to multiple CPU cores via the system bus.

Fig.2 BulletSystem.Update() runs and script data is cached.

Fig.3 Physics.Raycast busts the cache.

Fig.4 Transform.position= busts the cache again.

Now when the for-loop iterates we have to reload the script data and do everything again, for every bullet – thousands of times. You can see how cache misses pile up – we keep context switching between different data structures.

For a long time this was an intractable problem with Unity, but in 2017 the Job System was introduced to address batch-processing like this. I’ve suggested how this works in my illustration – we’re going to “schedule” work on other CPUs in Update(), and then gather their results in LateUpdate(). Each CPU will keep the different types of data hot in their respective caches, minimizing the cache miss rate.

Fig.5 By dedicating different threads to different jobs, we keep more relevant data hot in the cache more often.

Unity provides RaycastCommand for physics query jobs and IJobParallelForTransform for scene transformations. Unfortunately, the code for this is too complex for a code snippet in a blog, but here’s a abridged version to give you a sense of the implementation:


struct BulletTransformJob : IJobParallelForTransform
{
	[ReadOnly]
	public NativeArray<Vector3> positions;

	public void Execute(int index, TransformAccess transform) {
		transform.localPosition = positions[index];
	}
};

class BulletSystem
{
	const int CAPACITY = 4096;

	Bullet[] bullets;
	int bulletCount = 0;

	TransformAccessArray transforms;
	NativeArray<Vector3> positionsToWrite;
	JobHandle txJob;
	
	NativeArray<RaycastCommand> commands;
	NativeArray<RaycastHit> results;
	JobHandle physJob;

	void Awake()
	{
		/*
		Allocate all the buffer memory we'll need up-front
		*/

		inst = new Bullet[CAPACITY];
		transforms = new TransformAccessArray(CAPACITY);
		results = new NativeArray<RaycastHit>(CAPACITY, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
		commands = new NativeArray<RaycastCommand>(CAPACITY, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
		positionsToWrite = new NativeArray<Vector3>(CAPACITY, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
	}

	void Update()
	{
		if(bulletCount == 0)
		{
			return;
		}

		float dt = Time.deltaTime;

		/*
		Build job input buffers.
		*/
		for (int it=0; it<bulletCount; ++it)
		{
			var bullet = bullets[it];
			positionsToWrite[it] = bullet.pos + ( dt * bullet.speed ) * bullet.dir;
			commands[it] = new RaycastCommand(bullet.pos, bullet.dir, bullet.speed * dt);
		}

		/*
		Schedule a batch transform update.
		*/
		BulletTransformJob job;
		job.positions = positionsToWrite;
		txJob = job.Schedule(transforms);

		/*
		Schedule a batch of physics queries.
		*/
		physJob = RaycastCommand.ScheduleBatch(commands.GetSubArray(0, bulletCount), results.GetSubArray(0, bulletCount), 1);
	}

	void LateUpdate()
	{
		if(bulletCount == 0)
		{
			return;
		}

		float dt = Time.deltaTime;

		/*
		Wait for both jobs to finish, if they're still going.
		*/
		txJob.Complete();
		physJob.Complete();

		/*
		Handle bullet impacts, swapping with the end of the array to remove
		*/
		for(int it=0; it<bulletCount;)
		{
			var bullet = bullets[it];
			var hit = results[it];
			if(hit.collider == null)
			{
				bullet.pos += (dt * bullet.speed) * bullet.dir;
				++it;
				continue;
			}

			bullet.Release();
			HandleHit(hit);
			--bulletCount;
			transforms.RemoveAtSwapBack(it);
			if(it < bulletCount)
			{
				bullets[it] = bullets[bulletCount];
				results[it] = results[bulletCount];
			}
			bullets[bulletCount] = null;
		}
	}

	void OnDestroy()
	{
		/*
		Clean up after ourselves.
		*/
		transforms.Dispose();
		results.Dispose();
		commands.Dispose();
		positionsToWrite.Dispose();
	}
}

Just a quick look at the profiler in my stress-testing scene, and we’re comfortably above 100FPS. Most of the script time is actually spent on a quick-and-dirty prototype script that has nothing to do with bullets. 🤡

This isn’t even a build - this is in the editor, which is slow anyway.

I could push this further by simplifying the bullet objects themselves, or perhaps even rendering them with GPU Instancing as opposed to MeshRenderers, but for this early phase of development I’m happy knowing I don’t have any unsolvable problems which can come back to bite me later.

🖌️ Art Breakdown: Mecha Transformation! 変身!

Sun, 15 Jan 2023 00:00:00 +0000

Let’s take an art-dive into the inspiration and iterations on Tiny Starpilot’s transforming player jet/robot.

Like many venerable 3D games before me, my character started as a block of tofu.

physical-character control + map-splines integrated into a spline-flying state :) #gamedev #ScreenShotSunday pic.twitter.com/zPfcGAqqZG
— Max! (@xewlupus) November 18, 2019

I’m particularly proud of the squish on landing.

This design actually does a lot of lifting early-on. It cuts the overhead of iterating on character root motion, and clarifies little details like how when the tofu flies it can lean-forward 90° and point it’s local-up-axis forward to avoid having to swap from a tall to a long hitbox.

There’s a vocal contingent of indies who argue that this is also good to ship. After all, if the game is already fun as a greybox, what function does the art serve, except to demonstrate that you’re a big company with money to spend?

I don’t buy this. Even on ludological terms, there is utility in the semiotics of artwork for design-intent-signalling.

But indie games are more than mechanics – they’re a parasocial experience amongst and between creators and players. I’m not just building an experience to tickle your proprioception-pleasure-centers, I’m expressing solidarity by positioning the work within a shared enthusiast genre: stylish anime mecha shiiiit.

I could watch “booting up cockpit computer” sequences all day.

The more specific proximate motivation for me to investigate character art was seeing the cool process-posts and sizzle-reels Ivy May made for Coquette Dragoon using Blender. Now I wanted to learn Blender!!

Girls love + alternative mecha VN series Coquette Dragoon is now out in early access! ~50k words. Contains full first two chapters and preview of chapter three! Coquette is for adults only and contains dark/suggestive themes, read at your own discretion. https://t.co/mh62zNIE0h pic.twitter.com/nnBO7ZuKVu
— iby (@burgeroise) July 19, 2022

Support independent creators!

My first concept sketch was a vague mix between the Vic Viper from Zone of the Enders: 2nd Runner, and the eponymous Ikaruga Fighter.

I vocalized a lot of transformation and impact sounds while drawing this.

finished the arms and shield-panels. now just for the wing panels which I... forgot to draw in my reference images -__-;; pic.twitter.com/6mYJDNb81o
— Max! (@xewlupus) March 9, 2020

TL;DR Review of Blender: great modeling workflow, but character rigging was a pain.

One afternoon of game-engine-noodling and IK-wrangling later, and we were in business.

Full integration of new character + posing + shooting + jet-transformation!! #gamedev #unity3d pic.twitter.com/4IV12m4jsO
— Max! (@xewlupus) March 30, 2020

Skull Squadron color scheme, Naturally.

This looked sicc, and left me feeling pretty chuffed, but shortly it became clear there were some dealbreakers.

The asset was too detailed, and didn’t match the environment.
It was too labor-intensive for other props.
The “jet” shape lacked personality.

So I went back to the drawing board and designed a simpler character that more naturally decomposed into a “boxy”/low-poly"/"fifth-generationy" model.

I still wanted to retain the the boyish/feminine figure.

For the transformation, I decided not to reinvent the wheel and takes the basic mechanics of the Strike Valkyrie from my one true love, Macross.

The legs swing into trusters, and the arms fold between them, centered under the chasis.

My personal innovation was to make the knee slide-up along a track on the back of the leg to form v-shaped thrusters, which exagerates the thrust-aperture-narrowing animation in game and makes the afterburner feel more responsive.

I didn’t reinvent the wheel, but I did put my fingerprint on it.

This was a triumph, I’m making a note here: huge success. I’s hard to overstate my satisfaction. It looks great next to other low-poly assets, it’s easy to make quick changes, and it has a lot of personality.

I think I did a Good Job!

Now for the really hard part: picking a name. 🤦

⚙️ Tech Breakdown: Third Person Cameras in Games

Sat, 14 Jan 2023 00:00:00 +0000

Experience has made me opinionated about implementing 3rd person cameras. People naturally, but naively, think about the camera as a second actor in the world, following the player around (like Lakitu in Super Mario 64).

Lets discuss an alternative perspective, where you consider instead the player on the 2D picture plane (with code!).

We can rebuild him. We have the technology.

A big shoutout here in the jump to the work of John Nesky and Mark Haigh-Hutchinson in contributing to my understanding of “gamatography.”

So first off, let’s state clearly what the goals of a 3rd person camera system ought to be, as plainly as we can:

We want to place the player on the screen.
We want to control how much of the screen is covered by the player.
We want interactive control over the view direction.

Let’s make these into parameters that our code can script.

The following code samples will use Unity’s C# API for reference, but these methods are pretty generic and portable to any 3D environment. All you need to do is make sure you can write to the camera’s position and rotation.

struct CameraParams {
    public Vector3 trackingPosition;    // 3D world position of the player
    public Vector2 framing;             // 2D screen position of the player
    public float distance;              // offset between the camera and the player
    public float pitch;                 // vertical camera tilt
    public float yaw;                   // horizontal camera pan
}

The Tracking Position will come from the game – typically you just take the player position and possibly apply a low-pass filter or critically-damped spring to smooth out little bumps.

You can also get pretty sophisticated with it. On Solar Ash we projected the player position onto the ground, so that the camera wouldn’t “track” jumps, and only applied vertical smoothing to clean up uneven terrain, but no horizontal smoothing which would cause the camera to lag as Rei zipped around at 45MPH.

The green line here shows the tracking position.

Framing is a 2D value in “Viewport Coordinates” where (-1,-1) is the bottom left corner of the screen, (1,1) is the top-right corner of the screen, and (0,0) is the middle of the screen. This tells us where to place the tracking position on the picture plane explicitly.

For a platformer, you probably want to be bottom-third when looking horizontally, center when looking-down on the player, and bottom-center when looking-up (possibly biassed a bit horizontally for an “over the shoulder” feel).

In Tiny Starpilot I actually counter-frame the player and the reticle. When you’re aiming to the right, the player is framed on the left and vice-versa. This creates a more dynamic feel and also gives more screen-space to see what you’re shooting.

Really pushing how much I can get stub in with just spheres. #gamedev #screenshotsaturday pic.twitter.com/TsN2PJjsvN
— Max! (@xewlupus) January 8, 2023

Distance is just the world-space distance from the player to the camera. Pretty normal platformer logic is to come in close when looking up and pull far out when looking down. You also might pull out for action scenes but pull in for narrative scenes to draw peoples attention (you can also do that with field-of-view, but it’ll make people sick 🤮).

Lastly, Pitch and Yaw are the Spherical Coordinates of your viewing direction. In an Orbit Camera, you can just drive these from the right-stick or the mouse-delta, making sure to clamp the pitch and wrap-around the yaw.

One advanced technique is to consider the vertical component of the cross-product between the view-direction and player-velocity. Scaling and adding this into yaw simulates a “leash” effect.

So how do we convert these parameters to the camera position/rotation? Math!

static void SetParams(this Camera camera, in CameraParams camParams) {
	// compute "local" offset relative to our view rotation
    var tanFOVY = Mathf.Tan(0.5f * Mathf.Deg2Rad * camera.fieldOfView);
    var tanFOVX = tanFOVY * camera.aspect;
    var localOffset = new Vector3(
    	camParams.distance * tanFOVX * camParams.framing.x,
    	camParams.distance * tanFOVY * camParams.framing.y,
    	camParams.distance
    );

    // compute position and rotation
    var rotation = Quaternion.Euler(camParams.pitch, camParams.yaw, 0);
    var position = trackingPosition - rotation * localOffset;
    camera.transform.SetPositionAndRotation(position, rotation);
}

Implemented as an Extension Method for UnityEngine.Camera, as a flex.

I’ll leave puzzling out the trigonometry as a exercise to the reader, but notice how we consider the field-of-view and aspect-ratio of the camera to correctly convert the viewport coordinates into a world-space parallax offset.

In addition to being a more natural parameter-space to script gameplay with, these inputs also form a better blending space for seamlessly moving the camera from one tracking-target to another. If you just Lerp() the position and rotation you have no guarentee that the tracking position will stay on screen, or that the horizon will stay flat.

The key insight here is that, given a camera and a tracking-position we can inverse-compute the camera parameters, then the save the difference (or delta) between the two parameter spaces, which we then add to our current inputs, but blend-out over time. That way the interpolation is in our intuitive picture-plane space.

static void GetParams(
	this Camera camera,
	out CameraParams result,
	in Vector3 trackingPosition
) {
	var position = camera.transform.position;
	var rotation = camera.transform.rotation;

	// tracking position
	result.trackingPosition = trackingPosition;

	// pitch/yaw
	Vector3 eulerAngles = rotation.eulerAngles;
	result.pitch = eulerAngles.x;
	result.yaw = eulerAngles.y;
    
	// distance
	var toTrackingOff = trackingPosition - position;
	var fwd = rotation * Vector3.forward;
	result.distance = Vector3.Dot(toTrackingOff, fwd);
    
	// framing
	var toCameraOff = position - trackingPosition;
	var parallax = Quaternion.Inverse(rotation) * toCameraOff;
	var tanFOVY = Mathf.Tan(0.5f * Mathf.Deg2Rad * camera.fieldOfView);
	var tanFOVX = tanFOVY * camera.aspect;
	var screenToWorld = result.distance * new Vector2(tanFOVX, tanFOVY);
	result.framing.x = -parallax.x / screenToWorld.x;
	result.framing.y = -parallax.y / screenToWorld.y;
}

A little more verbose, but not too bad.

With this in hand, we can now write the orbit camera blendout:

struct CameraBlend {
	public Vector2 framing;
	public float pitch;
	public float yaw;
	public float distance;
}

static void ComputeBlendout(
	this Camera camera,
	out CameraBlend result,
	in CameraParams newParams,
) {
	camera.GetParams(out var oldParams, in newParams.trackingPosition);

	// offset from the new target to the current view
	result.framing = oldParams.framing - newParams.framing;
	result.pitch = oldParams.pitch - newParams.pitch;
	result.yaw = oldParams.yaw - newParams.yaw;
	result.distance = oldParams.distance - newParams.distance;

	// yaw can wrap-around
	if (result.yaw > 180f) { result.yaw -= 360f; }
	else if (result.yaw < -180f) { result.yaw += 360f; }
}

static void SetParamsBlended(
	this Camera camera,
	in CameraParams camParams,
	in CameraBlend blendout,
	float progress
) {
	var multi = 1f - Mathf.Clamp01(progress);

	var result = camParams;
	result.framing += multi * blendout.framing;
	result.pitch += multi * blendout.pitch;
	result.yaw += multi * blendout.yaw;
	result.distance += multi * blendout.distance;

	camera.SetParams(in result);
}

Notice how we don’t need to adjust the tracking position – we’re entirely blending in the picture-plane now, so everything stays in view.

This article has gone long, so I’ll wrap it up. But just to leave you with some last widsom: understanding these algorithms is the first-step towards extending them. Placing Solar Ash’s camera on a firm foundation eased developing more complex picture-plane oriented tasks, like making line-of-sight collisions less disorienting, changing gravity, or my personal favorite: positioning two heads balanced in a box nicely for dialog.

A little mathematical flexing from #SolarAsh preproduction that I was pretty proud of: a method for positioning a camera -- given two heads and a not-blocked-by-UI viewport subregion -- that's balanced for an player-controlled view rotation. #gamedev pic.twitter.com/KXlwisvu2x
— Max! (@xewlupus) December 21, 2021

👷 Project Update: Introduction to Tiny Starpilot

Tue, 10 Jan 2023 00:00:00 +0000

I juggle a dozen little side-projects which I dust off from time to time.

Tiny Starpilot was a minigame I conceived of years ago when I was freelancing between jobs, and considering the possibility of becoming a one-man-band mobile game developer.

The design was simple: hold the phone sideways and slide your thumbs along the side to act as tank-tread controls. Your ship would autofire on a regular beat.

Pew! Pew! Pew!

I scribbled some quick & dirty pixel art, downloaded some public-domain music, cooked up some bloops with bfxr, and scripted it up in Unity in an afternoon.

I was fortunate to participate in a San Francisco co-working space at the time – camped in a generously-donated spare-room at Double Fine – surrounded by other talented creators. I received feedback to use a 3D art style, since sales prospects were better that way. So I slapped some cubes together to make a more dynamic demo.

I thought the green guys could be, like, Space Squids.

I didn’t end-up going the solo-dev route, because I was recruited to join the amazing team at Giant Squid to collaborate on ABZÛ. It was a life-changing oppurtunity where I had a chance to grow as a programmer and as a technical game designer.

In particular, I learned a lot about 3D Camera Design and Character Control. I grew ambitous about making my side-game Full 3D, and not just “2½"D.

Handbreak drift, scatter-spawned asteroids, abzu-stule unrolling, player craft collision #screenshotsunday #gamedev pic.twitter.com/G2N8h45sYh
— Max! (@xewlupus) February 19, 2018

Asteroids, eat your heart out.

The idea for “drifting” came from conversations with my colleague Pete Angstadt as an alternative to the “reverse shooting” that wasn’t possible with the transition from tank-tread controls to a 3D flight yoke.

By this time ABZÛ had wrapped, and I’d transitioned to the new project at Heart Machine which would become Solar Ash. I was doing a lot of research into procedural animation for the Level-Scale Remnants, which bled into my silly side-projects, too.

strong "boston dynamics big dog" energy #gamedev pic.twitter.com/x5mPYynaqa
— Max! (@xewlupus) November 28, 2019

Tofu-Bot 4000

I’m a life-long 80s mecha anime obsessive, so instinctively I thought: if I merged this with Tiny Starpilot could this be a Macross fangame??

Transformation! #screenshotsunday pic.twitter.com/9GpxvkCnfn
— Max! (@xewlupus) December 9, 2019

Just call me Shoji Kawamori

After posting low-key incremental updates sporadically under the #screenshotsaturday tag for a few years, suddenly this blew up:

Really pushing how much I can get stub in with just spheres. #gamedev #screenshotsaturday pic.twitter.com/TsN2PJjsvN
— Max! (@xewlupus) January 8, 2023

3k likes in 24 hours on my low-follow feed probably represents the majority of engagement I’ve ever received on social media cumulatively in a decade, so I was (to say the least) surprised and humbled. That’s when I decided to start this devlog.

I hope that by posting more in-depth footage and generating interest, it’ll light a fire under my butt to update more regularly, and bring the dream of a public release closer to reality. 🤞

I envision posting in a few different flavors:

#update to track the game itself, with footage of new features
#tech for explainers on math, programming, and nerd shit
#art for concept sketches and 3D models I’m noodling on
#design for thoughts on action, iteration, and experimentation
#research for reviews of related games or anime that I’m studying

Thank you for reading my story, and following along in the future. There’s an old bit about how gamedev is like telling a joke and waiting for years to hear if anyone laughed. Your attention and engagement in the process motivates me to do the labor of pulling this stuff out the neoplatonic playpen of my imagination and presenting it online.

About

Mon, 01 Jan 0001 00:00:00 +0000

I’m a professional game programmer in Los Angeles. I’ve collaborated on MMOs, branded games, mobile games, micro consoles, and some prestige indie-games: ABZÛ, Solar Ash, and Hyper Light Breaker. I developed internal tech at Yacht Club Games, and now work as a freelance game/animation programmer.

Little Polygon is the development blog for my side-projects, and to pontificate on various gamedev-related subjects. You can learn more about me at my homepage.

If you’d like to reach out, you can find me on twitter, bluesky, mastodon, or email max at tinygalactic dot com.

Little Polygon Game Dev Blog

🏗️ Scaffold Level Editor

Background: Convex Decomposition

Navigation

Collision and Navigation in General-Purpose Game Engines

Looking Forward by Looking Backward

DOOM: Binary Space Partitioning

DESCENT: Convex By-Construction

Nightshift Galaxy Scaffold

⚙️ Tech Breakdown: AI with Finite State Machines

Algorithms for Game AI

C++ State Machines

⚙️ Tech Breakdown: Fixing Unreal's Horizontal Field of View

Indie Game Pitching in 2024

Introduction

How Do You Pitch a Game?

Creating the Demo

Creating the Deck

Pitching at Conferences

“Survive to 2025”

Future for Nightshift Galaxy

👷 Project Update: Tiny Starfighter

⚙️ Procedural Animation: Locomotion (Part 2)

⚙️ Procedural Animation: Locomotion (Part 1)

⚙️ Procedural Animation: Trailing Joints

⚙️ Procedural Animation: Inverse Kinematics

⚙️ Tech Breakdown: Collision Sliding

🖌️ Project Update: Playable Pilot Concepts

The Rebel Ace

The Imperial Commander

The Space Pirate

⚙️ Math Breakdown: Anime Homing Missiles

Iterative vs. Closed-Form Motion

Simulating Homing Missiles

Adding Noise

⚙️ Tech Breakdown: Bullet Hell

Memory Problem 1: Allocation and Object Pooling

Memory Problem 2: Context-Switching and Data-Oriented Design

🖌️ Art Breakdown: Mecha Transformation! 変身!

⚙️ Tech Breakdown: Third Person Cameras in Games

👷 Project Update: Introduction to Tiny Starpilot

About