There are several reasons why such inconsistencies happen. Take Ktor, for example: Until recently, the project belonged to a very different department than Kotlin itself. Compose Multiplatform, on the other hand, has good reasons matching the versions from Google’s Jetpack Compose. And kotlinx libraries might just have suffered from a lack of defined conventions and the nature of such issues: They will never become the #1 top priority and always stay within the realm of ‘would be nice to address eventually’.

I previously invited the community to a debate about a proposal attempting to unify the versioning schema used by the Kotlin ecosystem. As of writing this article, there are two top contenders:

Strict SemVer
1.0.0-alpha.1, 1.0.0-beta.1, 1.0.0-rc.1

Google / AndroidX
1.0.0-alpha01, 1.0.0-beta01, 1.0.0-rc01

While some audiences prefer the ‘Google Notation’, seasoned library developers often argue for the ‘Strict Semver’. Interestingly, my very first draft of a versioning proposal, shared with the Kotlin department, also proposed this notation. It seems like a decent convention used by a very significant share of our industry. It has clearly defined semantics and ‘easy to understand’ rules. I fully agree with Martin Bonnin and would also call it a ‘great versioning scheme for Kotlin libraries’.

The ‘ideal’ diversion

I am really fascinated by the concept of ‘engineering emotions’, and I strongly believe that all good engineers eventually have to overcome many of their ‘guiding feelings’: The idealism trying to satisfy our sense of beauty and order, the subtle irritation we notice when reading code that certainly works, but somehow does not fit our philosophy.

I failed to manage my emotions when I initially proposed that the Kotlin Department adopt a strict SemVer (1.0.0-alpha.1) notation. I noticed the inconsistency within our company’s libraries when starting a new project and had to choose a versioning schema. After realising we do not have a written convention, I considered using the same schema as other libraries and noticed the lack of consistency, which motivated me to address this. I had clear and simple goals:

I wanted to find a versioning schema/notation which

• Works predictably for practical use cases (e.g., is handled well by Build Systems like Maven, Gradle, and Amper)

• Is able to convey compatibility semantics for all our projects

• Can be adopted by many projects to eventually increase the entire ecosystem’s consistency

And I found that ‘SemVer’ already defines clear rules for pre-release classifiers (alpha, beta, rc), and I consider those rules very beautiful. The fact that other ecosystems seem to converge on this notation made me very excited, and I got started writing up a proposal to the team, not knowing that what I had taken to be an ‘ideal’ would, very quietly, change my initial goals! I wanted to move our Kotlin ecosystem toward a more consistent, predictable state. My proposal, however, aimed to align Kotlin with a supposedly ‘industry standard,’ well knowing that many of our projects and key ecosystem libraries would not be able to adapt to it. The entire ‘androidx’ ecosystem of libraries, including Compose, a vital part of our Kotlin Multiplatform technology, has already settled on the ‘Google’/’AndroidX’ notation. My initial goals, therefore, cannot be achieved by my very own proposal, and this is not the first time this has happened. During my research, I found basically no practical differences between the two notations. Build Systems handle them well; both look good, and both have easy-to-explain rules. The worst part: While aggressively seeking good reasons to switch to an entirely new notation (SemVer), I even found situations where the Google notation objectively works better. I still ended up writing a proposal, ignoring my initial goals and even ignoring my own research. All in order to satisfy my own sense of beauty and ‘greater ideal’.

My Proposal 1.0.0-rc01

As I noticed how my emotions guided me into missing my very objectives, my fix is easy:
I’ll correct my proposal to use the ‘Google’ notation. It works just as well and can be adopted across all projects within the Kotlin Ecosystem Department (speaking Kotlin, kotlinx, ktor, …). The notation is well known by many of our developers and has some nice properties

Still ‘SemVer’ compliant

From the official guide

A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. Identifiers MUST NOT be empty. Numeric identifiers MUST NOT include leading zeroes

Note that only numeric identifiers are not allowed to start with a 0 (e.g. 1.0.0-0 ) would be illegal, whereas 1.0.0-alpha01 is legal and is compliant with ordering rules for up to 99 alpha releases (Again, real-world build systems will handle even more releases just fine)

When 1.0.0-rc01 is better than 1.0.0-rc.1

Please keep in mind that the entire section here is me falling into an emotional trap again: Post-Hoc Rationalization: Coming up with reasons why my decision makes sense, after making it, that is. I still include it, because it’s fun to talk about. While I acknowledge the trap, I am still convinced that the decision between the two styles is simple, since only one can justify the original goals

Naive Version Parsers

While ‘SemVer’ offers a robust set of rules, we have to acknowledge that many real-life projects may need to quickly parse version numbers and compare them. I have written such code many times to manage integration tests, depending on the Kotlin version, or to manage builds. Since most of those implementations are naive or even lazy, I am a little too embarrassed to link them; have mercy on me! Such code usually works by splitting the version by the . character, converting major, minor and patch to integers, then comparing -pre-release-classifier alphabetically. While I agree that the implementation is wrong, we have to give credit to Google Notations’ more lenient properties towards such mistakes!

In this case, 2.14.3-alpha.20 would be wrongfully evaluated as ‘lower’ compared to the -alpha.5 counterpart

This effect is also visible in systems that do not care about library versions at all, such as simple indices on webservers or even filesystems. Again: I am not saying that the notation, therefore, is better, but we have to acknowledge its strengths.

Our release philosophy

During a call about Google’s versioning schema, Aurimas Liutikas also mentioned that the notation more closely aligns with how we actually think about our pre-releases, and I can see a lot of beauty in this argument: The ‘SemVer’ notation might lead to interpreting pre-releases within a given maturity level as ‘rolling’. E.g. 1.0.0-alpha.1 only being the first published build with the alpha maturity and 1.0.0-alpha.2 being the second public build with alpha maturity. The alpha, therefore, carries the maturity semantics independently of the build number (.1, .2, .3, …). AndroidX, however, wants to convey a publication as ‘complete’. alpha01 therefore being more distinct from alpha02. This comes in handy, as AndroidX already provides ABI-compatibility guarantees for pre-releases!

Summary

I think the last two arguments presented do not matter much (but were fun to make). What matters is a consistent schema across the ecosystem that works and can be adopted by many projects. I will proceed with my proposal, using the Google notation. Multiple people within JetBrains will review this proposal. I will keep you up to date on any changes to it. Once approved, you can expect a public, official KEEP to be created to promote this proposal beyond JetBrains.

Sometimes finding a solution in O(n²) might be pretty awesome. Sometimes it's a respectable achievement. In my case, it was good enough to get started.

I am talking about the current ‘project firework’ aka. “Compose Hot Reload”. As presented in some internal talks or YouTube videos, the crucial part of implementing Hot Reload is figuring out which parts of your application are dirty after some code was changed. Luckily, Compose is a functional framework, so we mostly care about functions (or some scopes within a function, to be more precise)

For the sake of this blog post, some simplifications will be made:
Let’s accept the fact that Compose Hot Reload has the concept of ‘scopes’ inside functions and a way to identify them. We also assume that Compose Hot Reload knows the entire state of your application before any request to reload classes arrives. Knowing a scope also means knowing a ‘hash code’ of the code inside the scope. As a simplification, you can think of this hash code as the hash of all bytecode instructions. I will present more details on the exact calculation of this hash in some other materials.

In the image above, you can see an example of the tracked runtime, where each scope gets its associated hash value. Note: The hash value includes only the code of the scope, child scopes are not included! This can then be represented as a tree/graph:

(Those graphics skills are unmatched, I know).

We can represent two kinds of relations: Either one scope is just a child of another scope
Or one scope has an explicit dependency on another scope (e.g. by calling a function).
In the example above, the code calls into other @Composable functions such as Button, Text, or Column as well as some utility function fontSize()

O(n²): good enough, until it is not

So here is how Hot Reload, before 1.0.0-alpha03, figured out which @Composable scopes are ‘dirty’ and have to be invalidated.

The first ‘n’: Number of @Composable functions/scopes

It’s fair to assume that in user-facing applications, the number of @Composable scopes grow linear to the size of the codebase n.

Hot Reload knows the entire state of the runtime before and after the reload happens. We take each known @Composable scope and perform a diffing algorithm. For the sake of this example, let’s assume we do change the fontSize() function to now return 72.sp instead of 48.sp.

As readers, we know that this means that the App$Column scope shall be marked as dirty

This becomes pretty clear when looking at the tracked runtime information presented as a tree:

All @Composable scopes are marked with the small Compose logo^.
In the example, there are four of them! App, App$Column, MyButton and content. After reloading arbitrary code, any of those scopes could get dirty. There, each scope will be enqueued to check for dirtiness.
But wait, we agreed that the number of @Composable scopes is linearly proportional to the entire codebase in Compose projects. We found the first n 😎

Who is dirty?

Since App$Column calls into the fontSize() function, we know that it has to be marked as dirty, should drop its UI state and re-render. However, we know that keeping the state in other @Composable scopes is totally fine. We can conclude that dirtiness follows only dependency edges and does not cross Child/Parent relationships.

Each scope will now calculate a new hash code, called the “Invalidation Key” by including all transitive dependency scopes. For the sake of simplicity, let’s also assume that dependencies between @Composable functions are also to be ignored.

The second ‘n’: Graph traversal

The example above seems easy: Again, for each @Composable scope, we will calculate this new hash code by traversing all transitive dependencies. The above image looks simple and flat because the graph we took as an example is simple: Only a single dependency from Appp$Column to fontSize() which we had to traverse. However, even with this simple code example, the ‘real life’ graph will look much, much more complicated. You see that the fontSize() method also depends on the .sp extension property, the getter of which will bring in more and more dependencies as well.

But how much work will this be for each scope to traverse its dependency graph? Certainly, this depends a lot on the structure of the code and its dependencies. It’s hard to predict how deep the dependency graph will be for each scope. Let’s do a field experiment:

Let’s run a huge application, such as IntelliJ itself, setup Hot Reload, perform a code change, and measure how long the resolution of all scopes will take:

Let’s change a simple String literal and reload: 59ms
Let’s perform some dummy changes in a base module: 53ms

That does not look too bad, right? Unless we realize two things:

1. When doing the experiment there were only 300 @Composable scopes loaded and the time it will take this algorithm will grow linearly with the number of those. Even having 30K scopes would not be considered a “huge” app and we can expect the algorithm to take ~5s, which is a deal breaker. Note: 30K scopes does not mean 30K @Composable funcitons. Each function, especially when using control flow, will have multiple child scopes.

2. Those results were only reached by manually disabling one rather important kind of dependency edge:
   What happens if a function depends on an interface function, or any open or abstract function for that matter? Then it certainly shall be marked dirty when any of the implementations change. This, however, will bring in many, actually unnecessary nodes in our dependency graph to traverse.
   Imagine depending on an interface MyInterface and it has 100 implementations. You changed one and now all those 99 unnecessary implementations will be part of the dependency graph (and their dependencies as well!). But what happens if those implementations rely on other interfaces with many implementations themselves? This is what I called the ‘super class bomb’. When this happens (and this happens in large projects), then the dependency graph, for each @Composable , is proportional to the size of the code base. Jeez, here we go: We found the second n , did we?

Let’s re-enable this part of hot reload (virtualMethodResolve) and test IntelliJ changes again:

• simple String literal and reload:
  I had to cancel this test, because I am impatient as hell. However, my estimate from the progress bar would have been: 225 Minutes 💣🤯

• some dummy changes in a base module
  This one, I was patient enough to wait through: 8m 28 💣 🥺

Note, I was unable to come up with a theory as to why the first ETA is so much higher than the second result. From what we have described about the algorithm, so far, it should not matter where the change is located: We’re doing the same work after all. My best theory is that there might be just one bomb, which takes almost all the time. Maybe If I would have waited just a little bit longer, then we would have seen ~9m as well?

Fighting ‘n’ and ‘n’

The ‘two hashes’ idea, the first hash being only the ‘semantic hash’ of the scope in isolation, the second hash being this ‘invalidation key’ was pretty elegant imho. It served me pretty well in understanding how the system behaved and made it easy to debug. However, we’re certainly doing a lot of work that we do not care about: Who cares about the hash code? All we actually care about is whether or not it has changed or not, right? Equally: When trying to find what is dirty, it seems like the old algorithm was searching for the needle in the haystack… 👀 after throwing it in itself.
See: The runtime knows exactly which classes changed to begin with because the user sent a request to reload a set of particular classes! We know that nothing else could have changed. This information was not used in the algorithm. The direction of following from interfaces to their implementations also is unfavorable. Methods might have many implementations, but we can think of an implementation mostly having one super!

Turnaround: 🎶 Every now and then I get a little bit terrified… 🎤

Let’s build another algorithm, that does not start at the @Composable scopes, but at the classes which the users requested to reload. This number is rather stable (and certainly does not depend on the size of your project, n). We can identify the scopes, inside the ‘to be reloaded’ classes as dirty or not, by comparing their hashes against the previous ones. From here, we do the same walk, but in the inverse direction: We start from what is dirty, find scopes that depend on the dirty scope, mark them as dirty, and repeat!

Another crucial difference: Even though we might initially enqueue multiple scopes as ‘dirty’, we still only have to traverse the dependency graph once, instead of doing it for each starting point (previously @Composable scope). One ‘n’ seems to be killed! How about the number of nodes and vertices in this dependency graph? The static dependencies between code stays still the same, however, we drastically reduced the number of edges because of the 1:m relation between super and subclasses! We can straight up follow into the super method and mark it as dirty (compared to walking from one abstract method into all implementations!). This will help significantly, but theoretically, we can still have dependencies inside the code which might require traversing a huge part of the graph!

Turnaround: 🎶 Every now and then I get a little bit nervous … 🎤

Let’s face some emotions: Yes, I am proud that changing an implementation of an interface on which a static field depends, which is used in a @Composable function depends upon will refresh the @Composable . It’s awesome, it’s a nice brag and it makes for cool demos. Simply speaking: I am proud as a developer of how predictable this system would work. But is me being proud of fancy demos really worth following this graph into 15 levels deep? I mean: If the code change did not affect the @Composable scope directly, nor any immediate dependencies of this scope, then it’s pretty fair to say that the state kept in his scope is still OK. No need to invalidate! Therefore, we can stab the remaining potential n by just limiting our ‘mark dirty’ graph walk to a certain depth and calling it a day. We walk 3, 5, and 10 edges: If we have not found any @Composable scope, then fine: Let’s keep it!

Result

There has been a small benchmark written for this composable:
It will compile 1K or 10K synthetic Kotlin source files, which depend on each other, and perform a reload of 10 or 100 classes.

This benchmark at least confirms our suspicion: The number of total classes is irrelevant for this algorithm (first n). If we also accept the fact that during typical development flows the number of classes reloaded is pretty stable and does not depend on n, then we could say that we fully eliminated n: Total number of classes in the project from the equation!

But synthetic benchmarks aside:
Let’s spin up IntelliJ and make the same changes, which previously provoked the ‘super class bomb’!

• simple String literal and reload:
  previous (ETA) 225 Minutes 💣🤯 → new 500 us

• some dummy changes in a base module
  previous 8m 28 💣 🥺 → new 18 ms

The new algorithm will be available on 1.0.0-alpha03 , which also reduced the memory footprint of the runtime-tracking significantly.

In this blogpost, I would like to share my new favourite system property, how it can be used for significantly improving DX and thank the team who invented it in IntelliJ.

And who knows, maybe other engineers will share even cooler techniques in the comments or follow-up blogposts as well? 👀

My Fault

One of the projects where I was not too happy with the DX, debugging broken tests was set-up by my very own...

In my previous blogpost, introducing 'Poject Firework; AKA Compose Hot Reload', I proudly talked about my screenshot tests and how quickly they made me confident about the code. But did I tell you about the ceremony of debugging those tests? I mean, of course it's easy to just press 'Debug' in IntelliJ on the test's run gutter and hit a breakpoint in the code that executes the test, but…

Some Tests will launch new processes

** which will launch new processes, which will launch a new process, …

My project's main component is a javaagent, which can be attached to Compose applications, making them hot-reloadable. Integration Tests will run against different versions of Gradle, Kotlin and Compose. So typically, the @Test (or @HotReloadTest in my case) annotated method will just coordinate launches of new processes to test the entire build of a project.

This means, when I click the test-run-gutter, IntelliJ will

1. Launch a Gradle Daemon (if necessary) | JVM #1

2. The Gradle Daemon will launch a JVM to execute the tests within (Test Engine) | JVM #2

3. The Test will decide which parameters (e.g. which Gradle version) to run against and then create the test project, launching a new Gradle Daemon | JVM #3

4. The Test Gradle Daemon will then launch a test application which can be used to run the test against | JVM #4

If there are issues (e.g. a failing test, or god forbid: a bug 😳), the problem is mostly in the code of the 'javaagent', however, this code runs in the 4th JVM in my chain of sub-processes! While pressing 'Debug' in IntelliJ will stop breakpoints in the test infrastructure code (JVM #2), it won't affect the part I actually care about the most 🥺

So, when I had to debug a given test, I had to start a separate 'Debug Server' first and instruct JVM#4 to connect to it; However, this was also not ideal: Either I made the debugger connection suspending, this would fail the test if no debug server was started before. A non-suspending connection can be added to the JVM invocation unconditionally, but then my 'premain' function might run before the debugger connection is established. Therefore, I decided to introduce a @Debug annotation that a developer should add to the test, after launching the Debug Server manually (cough, cough*: 😩* Ceremony! 😩*)*

There has to be a better way: My new favourite System Property!

It is clear how the ideal solution would work: Press Debug in IntelliJ and get the breakpoints in the sources you care about engaged, ready to debug!

And wait, how does the IntelliJ debugger even know about the JUnit Test Engine JVM? What kind of magic will connect IntelliJ -> Gradle -> JUnit?

The answer is the System Property I have teased:

🎉 idea.debugger.dispatch.port

And I think the name is almost self-explanatory!

When running in 'Debug' mode, IntelliJ will launch a special 'Debugger Dispatch' server and inject an init-script into the build process. Before each 'JavaExec' task (tasks which will launch a new JVM) is executed, the task will connect to this 'Debugger Dispatch' socket and a new 'Debug Server' will be provisioned. Once the 'Debug Server' is started, the new JVM launches with suitable JVM args to connect to it.

And we can re-use the same mechanics to dynamically provision a debug server from IntelliJ for any other purpose! No matter where our code is running. All we need to do is: Check for the presence of the idea.debugger.dispatch.port System Property, connect to the socket and request a new Debug Server!

The protocol is straightforward: It uses java.io.DataOutputStream and expects three Strings being sent through the pipeline.

1. The 'Debugger ID': This will always be Gradle JVM in our case

2. The name of the process: This can be picked freely

3. Arguments (In our case we can just supply DEBUG_SERVER_PORT=port

Here is an example function which will find a free port, connect to the 'Debugger Dispatch Server' and request a 'Debug Server' being started. Once a single byte is received in response, we can assume that the 'Debug Server' is launched and we can proceed to start a new JVM with the debugging arguments.

And just like this, all the ceremony can be brought into core test-infrastructure, and a developer can just press 'Debug' and it just works:

___

Thanks to the IntelliJ Java Build tools team who introduced the 'Debugger Dispatch Server'

Video Version: https://youtu.be/I_FMnPaEBEA

Gullible me; I posted a screen recording into the #kotlin-random Slack channel on October 8th. Alongside this screen recording, I claimed:
"I think it would be easily doable to build a stateful hot-reload if*[…]*".

I wanted to create educational content about ClassLoaders and thought it would be great to showcase building a hot reloadable UI for Compose.

The if part in my initial message was much bigger than expected (who would have thought). But the demos sparked a lot of interest, and with the motivation and many good technical suggestions from our colleagues, two months later, a promising prototype is available.

This blogpost will explain the current state of said prototype (with a few demos), tries to give some technical insights on how it works and finally shares the plans on moving forward.

The current state of Compose Hot Reload

Prototype: https://github.com/JetBrains/compose-hot-reload

The state of the current prototype can be quickly summarized as:

• Almost all changes to your code are allowed, except removing supertypes from a class (see DCEVM part later)

• Changes in bytecode associated with a given "scope" in Compose will lead to this scope being invalidated (remembered values will be re-calculated)

• Gradle is fully implemented, Amper is 'work in progress', Support for the IntelliJ build will be next

Demo 1: Simple Literal Change

This is the 'Hello World' of Hot Reload; Notice the reloading indicator getting yellow once the compilation starts and turning green after the reload is finished.

Demo 2: Simple Layout Change

As it was promised earlier that almost arbitrary code changes are possible, I expect this demo to be almost boring. The UI switches the Layout from a Row to a Column and back.

Demo 3: Add State and change the interaction with it

Now it seems to get more interesting. This demo will introduce a 'remembered' state (count), adds a Button which will increase this count and add a Text to render it. You can see that pressing the Button will increase the counter on screen. Changing count++ to count++ will change the behaviour of the Button and the counter gets decreased afterwards.

Demo 4: Adding a new function

This Demo shows the power of our JetBrains Runtime. Adding this new 'NewComposable' function is not a problem. Traditionally, such changes are not permitted in the JVM world.

Demo 5: Rename a function (ABI changes)

This demo also shows a typical limitation of hot reloading being lifted (thanks to the JetBrains Runtime). Renaming the 'NewComposable' function to 'NewName' is possible. Adding the :) afterwards reveals that the hot reload is still alive.

Demo 6: Adding a new Class

A new class 'NewClass' is added and rendered as a String. You can see the default .toString() method rendering the instance after reloading.

Demo 7: Adding a new property to an existing class

One of the first questions people do have with such a potent hot reload implementation is "What will happen if we add a field/property to an existing class?" . The demo above shows the App Composable being invalidated, a new instance being created and rendered. This is the easier case. The JetBrains runtime also supports keeping previous instances when adding fields. In this case the JVM will fill new fields with the JVM defaults (null, 0, false).

Demo 8: Compose Hot Reload understands code dependencies!

This one makes me very proud; however, it is a little bit tricky to see what is going on here.

Compose Hot Reload understands which code (and its associated Compose Groups) depend on other functions and 'scopes' (more technical explanations can be found in later sections).

In this demo, we can see the App Composable picking a random value on the first run and remembering it, even retaining it in later changes. This random value is produced by the calcualteMagic function. Now look close! The App Composable gets invalidated, and a new value will be picked by us changing the body of the calculateMagic function. The runtime will track which code is changed and will resolve all Composable groups which are supposed to be invalidated.

Demo 9: Reload Status Indicator

We all have been there once in our professional life: Debugging for hours on end, starting to doubt our entire model of the universe, just to find out that our code was not even running at all. This is a very unpleasant experience and one of the reasons why developers might lose trust in ‘hot reload technologies’. In this demo, you can see a status indicator glowing red, next to your application, making it obvious that your code was not reloaded.

Once the compilation error was fixed, the indicator jumps from red to yellow back to green.

Demo 10: 'Dev Tooling': Clean Composition

The floating 'Compose Icon', following your application, can also be pressed to open advanced tooling. This is tooling is rendered in a separate process, which can be helpful to recover your application from a very broken state. In this demo, the 'Clean Composition' button is pressed, which will clear all remembered states of the UI and cancels all effects.

Implementation Details / Technical part

Overview

The prototype contains several conceptional parts:

Build Tool Integration

While there is an Amper integration currently 'work in progress', lets focus on the Gradle implementation for the sake of simplicity.

https://github.com/JetBrains/compose-hot-reload/tree/master/hot-reload-gradle-plugin/src/main/kotlin/org/jetbrains/compose/reload

The build tools job is to launch a given Compose application in a special 'hot' mode. This requires

• Ensure a JBR is available (download if necessary)

• Set necessary JVM arguments to allow redefining classes

• Attach and setup the 'Compose Hot Reload Agent'

• Provide and 'engage' the 'Compose Hot Reload Runtime'

• Provide instructions to the application on how to file a 'recompile request' back to the build tool.

Orchestration Server

https://github.com/JetBrains/compose-hot-reload/tree/master/hot-reload-orchestration/src/main/kotlin/org/jetbrains/compose/reload/orchestration

We will see later that multiple processes are involved in 'orchestrating' the hot reload application (Dev Tooling Process, Recompiler Process, optionally IDE tooling, …). Those processes communicate through a simple protocol which allows sending messages (OrchestrationMessage) as broadcasts to all participants. If not hosted outside (e.g. by the IDE), the 'Hot Reload Agent' will start this Orchestration Server and tell child processes how to connect to it.

Compose Hot Reload Agent

https://github.com/JetBrains/compose-hot-reload/tree/master/hot-reload-agent/src/main/kotlin/org/jetbrains/compose/reload/agent

The agent is responsible for setting up the Hot Reload environment. It will try to find the Compose Runtime and enable a 'Hot Reload Mode', which was initially intended to be used for Android (Live Edit). This mode will allow runtime errors to potentially be recoverable.

If not hosted somewhere outside (e.g. by an IDE plugin), the agent will start the 'Orchestration Server' and launch the 'Dev Tooling Process' as well as the 'Recompiler Process'.

A hook into the JVM will be installed. This hook will allow analyzing the actual bytecode when initially loading a class and when redefining it. This 'runtime analysis' will then know which 'Compose Groups' to invalidate by looking at changes to the bytecode.

The Agent will handle 'ReloadClassesRequest' messages, sent from the 'Recompiler Process', using the JBRs implementation of DCEVM (Enhanced class redefinition for Java)

Runtime Bytecode Analysis

Let's look into how the Compose Hot Reload Agent knows which states / Compose Groups to invalidate. We'll pick a simple example, where @Composable fun Foo has a simple remembered state, a Button to interact with said state and two utility functions.

When initially loading the class containing the Foo function, the Agent will track the 'Runtime Information'. After building 'Compose Aware' bytecode tokens, all functions will be parsed into a 'Runtime Tree'

This 'Runtime Tree' will understand the bytecode in functions and builds the corresponding scopes (called 'Compose Group'). You can see that the simple example will start a RestartGroup in the Function Foo. The tree will also include branches in your code.

Once all scopes/groups are parsed, the Runtime Info will be constructed, assigning each scope a 32-bit hash value. This hash value is almost the hash of the bytecode inside the given scope. For obvious reasons some parts are ignored (such as line number depending constants/instructions). If this hash changes, we can assume that the body of the function changed significantly enough to invalidate the group.

This 'Runtime Info' also tracks dependencies between functions!

See how Foo depends on FooKt.paddingValue and FooKt$Foo$2 (lambda inside the Button {} call) depends on FooKt.buttonText

When classes are 'reloaded'/'redefined', the Compose Hot Reload agent will calculate which Groups to invalidate. Another 'key' will be resolved, the 'Invalidation Hash Key'. This hash is calculated by creating a compound hash of a scope with all its dependencies included (transitively). If such an 'Invalidation Hash Key' changes after reloading classes, the associated Compose Groups will be invalidated.

Note

This obviously requires the actual Compose Group Keys to be stable during a reload. The Compose compiler required three patches to guarantee stable enough keys. Those patches will also improve the reliability of other tools (such as Google's 'Live Edit'). Two out of those three patches landed in kotlin.git/master (compose compiler). The third patch is currently pending. Therefore, custom 'firework' builds of the Kotlin/Compose Compiler are required when using the prototype.

Engineering Goode: Look Ma, Screenshot Tests with Hot Reload

Testing a system across multiple processes is tricky. Gaining confidence of Hot Reload really requires some kind of 'visual verification'. Screenshot tests, for example, will host the Orchestration Server in the test process, compile provided source code, spin up a special 'Application Under Test' and communicate with the Application Process by sending 'Orchestration Messages'.

The 'initial source code' here will define an application with a simple counter-state, which will render a text Before: 0 initially. You can see that the 'application under test' allows reacting to 'TestEvents'. Receiving a 'TestEvent' will increase the counter.

We'll take an initial screenshot, send two events, verifying the counter works by taking screenshots 'before-1' and 'before-2'.

Finally, the text "Before" will be replaced with "After" in code and the application reloads hot. The final screenshot will verify the state being retained (by rendering After: 2)

A test failure would then show up as:

Future Plans

The time since October was mainly used to validate the concept, trying to find as many issues as possible, trying to find any good reason not to continue. There was plenty of sweat and blood put into this project (although, no tears, because I am a brave engineer, not even a little, I promise 👀). My goal was to finish this experimental stage before new year.

I am happy to announce that the project is moving into its next phase: Stabilization. 🎇

Stabilization

Stabilizing the JBR implementation of Hot Reload requires checking the following boxes

(Next to the obvious part of finding bugs and issues and fixing them)

• All patches into the Compose Compiler need to be available in a stable Kotlin Release

  • Target: Kotlin 2.1.20 | Potentially Kotlin 2.2

• Ide Plugin is available

  • Currently prototyped by @Konstantin Tskhovrebov

  • Allows starting 'DevelopmentEntryPoint' annotated @Composable functions

  • Hosting the OrchestrationServer + ensuring that reloads are explicit by the user (CMD + S)

  • Provides the JBR, no need to download another one.

• Support for IntelliJ Plugin development (Optional, but very desirable)

  • The IntelliJ Plugin Gradle plugin integrates Compose Hot Reload. Allowing for a very smooth plugin development experience.

• Support for IntelliJ development in ultimate.git (Optional, but desirable, help needed!)

  • When working on IntelliJ inside our monorepo with Compose, allowing run configurations to start in 'Compose Hot Reload Mode' would be great.

Dreaming Bigger 1: Kotlin Preview JVM Compiler

The first stabilization focuses on the JVM and the JetBrains Runtime in particular. However, there might be many projects which might only target Android or (Android + iOS). Adding a JVM target to the project is doable but requires restructuring code and is not particularly cheap (especially for larger projects). One of the bigger pain points might be providing all 'actual' implementations on the JVM which have been defined in common code.

A special 'JVM Preview Compiler' is imagined, which is able to take your common code, producing executable JVM bytecode. Missing 'actuals' can be mocked! If your hot reload session encounters an actual implementation, which has not been defined for the JVM, then certain levels of gracefulness can be used.

Missing actuals returning Unit or nullable types could return null or Unit .

A placeholder box@Composable could replace missing functions @Composable functions.

Exceptions can be thrown otherwise.

All incidents, when your 'preview hot reload session' runs into a missing actual will be reported by the runtime as 'Orchestration Message', which could be picked up by the IDE suggesting to create the missing actual for you.

Note: This is a simple description of what we might want to work on next. I am sure that many readers will be able to point out many issues and flaws with the exact workings described above. Thats OK.

Dreaming Bigger 2: Next Targets | Android, iOS, Wasm

// To be continued…

Project Wizards can be great; they spare us from doing repetitive, boring, boilerplate tasks over and over again. However: Wizards are no replacement for education or documentation: I have seen so many engineers getting so used to only using wizards that they have no idea what their builds are actually doing or what certain blocks in their buildscripts are intended to do. Once shit hits the fan and something does not work out anymore, frustration builds up quickly, and tools like Gradle are blamed for the learned helplessness which has anchored itself over years and years.

This blog post is intended to "de-wizard" setting up Kotlin Multiplatform projects by explaining every step required to create a Compose app by hand (w/o using the wizards). Every step will be described in detail, and an on-screen video tutorial/commentary will also be attached.

ℹ️ You can also watch the full 'walkthrough' here:

https://youtu.be/fmFezt-2IBo

ℹ️ Alongside this post and videos, I also pushed this GitHub project, which contains the setup steps as individual commits in the 'main' branch:
https://github.com/sellmair/kotlin-multiplatform-getting-started

Create a new (empty) project / .gitignore

In the beginning, there was nothing. Let there be a .gitignore file, so we can start checking in our project into VCS. We're planning to use IntelliJ and Gradle for the Kotlin Multiplatform (KMP) app. I recommend keeping the .gitignore file simple

.idea
.kotlin
.gradle
**/build

• .idea: IntelliJ likes to check in configuration files here. For Gradle projects, most of the configuration is done within Gradle, so a Gradle sync should be all you need. I recommend checking in files from within .idea only if you really want to share them (e.g. some useful run-configurations or shared dictionaries)

• .kotlin: Used by the Kotlin Gradle Plugin (KGP) to store project-level intermediate files, or files produced for the IDE (by e.g. transforming dependencies)

• .gradle: Project-specific cache directory generated by Gradle.

• **/build: Gradle (and Gradle Plugins) conventionally put the build output in a build folder.

Creating the initial settings.gradle.kts

Creating the settings.gradle.kts file will get us started with using Gradle. The first piece we should configure is telling Gradle where to download libraries and plugins from. There are two top-level blocks to consider:

pluginManagement {
    repositories {
        mavenCentral()
    }
}

dependencyResolutionManagement {
    repositories {
        mavenCentral()
    }
}

The pluginManagement block will configure Gradle to download Gradle plugins from mavenCentral, whereas the dependencyResolutionManagement is configuring the repositories for downloading libraries.

Loading the Kotlin Multiplatform Plugin

We told Gradle from where to download plugins and libraries: Let's request the Kotlin Multiplatform plugin now. Gradle's plugin loading is not straightforward as it might isolate certain Gradle plugins from each other (in which case you might run into hard-to-understand and debug issues). One strange looking, yet effective way of avoiding such issues is to load plugins in the root build.gradle.kts . Using apply false will load the plugin and define its version for all 'subprojects' / 'submodules', but not apply them.

plugins {
    kotlin("multiplatform") version "2.0.0" apply false
}

Loading the Android Gradle Plugin

Similarly, the Android (Application) Gradle plugin can be loaded in the root build.gradle.kts file.

plugins {
    kotlin("multiplatform") version "2.0.0" apply false
    id("com.android.application") version "8.5.1" apply false
}

However, we will be unable to download the Android Gradle Plugin from Maven Central. We instead need to configure the google() maven repository additionally. Since Gradle will respect the order of repositories, trying to download plugins and libraries sequentially, I also recommend configuring the google() repository to only be used for .*android.* or .*google.* packages. This will decrease initial IDE sync and CLI build times, by avoiding many unnecessary network requests

repositories {
        google {
            mavenContent { 
                includeGroupByRegex(".*google.*")
                includeGroupByRegex(".*android.*")
            }
        }

        mavenCentral()
    }

Setting up the ':app' module / Kotlin Targets

In the settings.gradle.kts we can use the include(":app") function to register a "Gradle Subproject" (which is also often called a module in the community). To configure this module, creating a app/build.gradle.kts file is required.

Typically the build.gradle.kts file starts by loading plugins. In our case, we can start loading (and now actually applying) the Kotlin Multiplatform, as well as the Android Application plugin.

plugins {
    kotlin("multiplatform")
    id("com.android.application")
}

Note: Since we already 'loaded' the plugins in the root build.gradle.kts with a specific version, we are not required to declare a version here again. As we want to use those plugins in the. ':app' module, apply false is obviously also not required.

Kotlin is capable of compiling for many "Kotlin Targets": It is required to define the targets your code is supposed to build for. Since this guide is showcasing Kotlin for Android, iOS, and Desktop (JVM), the following targets will be declared:

kotlin {
    jvm() // <- for our Desktop app
    androidTarget() // <- Obviously to support Android

    iosX64() // <- Simulator for x64 host machines
    iosArm64() // <- physical iPhone
    iosSimulatorArm64() // <- Simulator for arm based host machines
}

Setting up the Compose plugins

There are two Compose (Gradle) plugins that we should load when using the Compose framework

1. The kotlin("plugins.compose") plugin will load the "Compose Compiler Plugin" for Kotlin, which will do all of the magic, transforming your code while compiling Kotlin.

2. The id("org.jetbrains.compose") plugin will set up your build to support packaging your application, managing resources, ...

Similarly to the Kotlin Multiplatform and the Android Application Plugin, it makes sense to load those plugins in the root build.gradle.kts and then apply them in app/build.gradle.kts

/* root build.gradle.kts */
plugins {
    kotlin("multiplatform") version "2.0.0" apply false
    kotlin("plugin.compose") version "2.0.0" apply false
    id("com.android.application") version "8.5.1" apply false
    id("org.jetbrains.compose") version "1.6.11" apply false
}

/* app/build.gradle.kts */
plugins {
    kotlin("multiplatform")
    kotlin("plugin.compose")
    id("org.jetbrains.compose")
    id("com.android.application")
}

Writing the first @Composable function

Before we can use the @Composable annotation from Compose, declaring foundational dependencies is required. Since the compose dependencies are expected to be shared across all Kotlin targets, we can use the commonMain source set to add those dependencies.

/* app/build.gradle.kts */
kotlin {
    sourceSets.commonMain.dependencies {
        implementation(compose.foundation)
        implementation(compose.material3)
        implementation(compose.runtime)
    }
}

Those compose.{xyz} libraries will be available in the build script since we have loaded the org.jetbrains.compose plugin. Adding those three dependencies will not only resolve the @Composable annotation, but would even allow us to use the material3 components (optional, of course).

The first composable can then be written in src/commonMain/kotlin/

package io.sellmair.app

import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.unit.sp

@Composable
fun MainScreen() {
    Text(
        "Hello from Kotlin!",
        fontSize = 48.sp
    )
}

Android: Creating the MainActivity

After we have created the @Composable function, we can wire Android up and create the MainActivity.

Code, specifically written for Android, can be placed into the src/androidMain/kotlin source directory. Here, we can just create a MainActivity.kt as usual.

However, since we want to use Compose on Android we have to perform several configurations:

Adding Android-specific dependencies

The androidMain source set can be used to add dependencies, specifically for Android. The activity-compose and appcompat libraries are recommended:

/* app/build.gradle.kts */
kotlin {
   sourceSets.androidMain.dependencies {
        implementation("androidx.activity:activity-compose:1.9.0")
        implementation("androidx.appcompat:appcompat:1.7.0")
    }
}

android.useAndroidX

Android requires to explicitly 'opt-in' when using androidx. Do not worry: If you forgot to do this, a nice error message with instructions will be printed.
To opt-in, the gradle.properties file needs to contain the following line

android.useAndroidX=true

Configure Android: compileSdk, namespace, applicationId

Creating an Android app requires some Android-specific configuration to be done. This includes choosing

• compileSdk: Which "Android version" you want to compile against (aka. which version of the APIs you want to see when coding)

• minSdk: The minimum "Android version" you want to support in your app

• targetSdk: Which "Android version" do you 'target' as in 'support all features of'.

• namespace: Under which package shall the 'generated' code from Android be placed

• applicationId: Unique ID for your application (suggested to be the same as namespace.

Creating the AndroidManifest.xml

Shipping Android apps also requires declaring an AndroidManifest.xml. The file can be created under app/src/androidMain/AndroidManifest.xml

The minimal setup, to create an app, is to

• Provide a 'label' (a.k.a. a name for your app)

• Select a theme

• Declare the activity

<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <application
        android:label="KMP Setup Sample App"
        android:theme="@style/Theme.AppCompat.Light.NoActionBar">

        <activity android:name="io.sellmair.app.MainActivity" android:exported="true">
            <intent-filter>
                <action android:name="android.intent.action.MAIN"/>
                <category android:name="android.intent.category.LAUNCHER"/>
        </intent-filter>
        </activity>
    </application>
</manifest>

Selecting a JVM Toolchain

When compiling code for Android we want to make sure that we consistently use one JVM toolchain. If not set correctly, we might be greeted with an error message like
"Inconsistent JVM-target compatibility detected for tasks"...

While some teams have more complicated requirements for their JVM toolchains, most projects are very well advised to just use one consistent jvmToolchain for their module. In the app build.gradle.kts do:

/* app/build.gradle.kts */
kotlin {
    jvmToolchain(17)
}

✅ After seeing the Composable on an emulator screen, we can consider the Android setup done.

iOS: Building the .framework

The architecture for integrating our Kotlin code into an iOS app looks something like

-> Compile Kotlin -> Build iOS .framework files -> Compile Swift -> Profit.

However, just declaring the iOS targets in the kotlin {} block will not yet build the .framework files from Kotlin. We need to configure the creation of those binaries for all iOS targets

import org.jetbrains.kotlin.gradle.plugin.mpp.KotlinNativeTarget

// ... 

kotlin.targets.withType<KotlinNativeTarget>().configureEach {
    binaries.framework {
        baseName = "KmpApp"
        isStatic = true
    }
}

In this (example) case, we would like to build a 'static' framework with the name 'KmpApp' for any "KotlinNativeTarget".

When building the app (e.g by invoking ./gradlew assemble) We should see the frameworks being located in app/build/bin

However, at this point it might be wise to increase the maximum amount of heap memory Gradle is allowed to allocate. Adding the following line to the gradle.properties would allow up to 6 GB of heap.

org.gradle.jvmargs=-Xmx6g

Creating the Xcode project

When creating a new Xcode project we can select iOS / App and put the project into our :app module (e.g. into an iosApp folder)

Create 'Compile Kotlin' run script 'phase'

As mentioned previously, we want to ensure that Kotlin produces its .framework before we compile our swift code against it. We can add a "Run Script Phase" to the Xcode "Build Phases" and call it "Compile Kotlin".

In this script, we are allowed to invoke the Gradle build to produce the requested .framework files. Using the embedAndSignAppleFrameworkForXcode Gradle task will allow the Kotlin Gradle Plugin to read the environment variables from Xcode, which will lead to building exactly the .framework for the 'configuration' currently selected by Xcode.

cd "$SRCROOT/../../../"
./gradlew :app:embedAndSignAppleFrameworkForXcode

Note: The ./gradlew invocation is prefixed by a cd command to change the current working directory to the root of the Gradle project (which also will contain the gradlew file. How you change the working directory obviously depends on the location of the Xcode project.

Disable "user script sandboxing"

Before we can test the Xcode build, we have to disable "user script sandboxing", as the Gradle build step is not supposed to run in a 'sandbox' as it's a grown-up part of our build chain now.

Adding our .framework to the 'Framework Search Paths'

To compile and link against our Kotlin code (from within Xcode), we need to add the locations of our .framework files to the 'Framework Search Paths'.

$SRCROOT/../../build/xcode-frameworks/${CONFIGURATION}/${SDK_NAME}

When using the embedAndSignAppleFrameworkForXcode, the expected location of the framework is inside the 'build' directory under 'xcode-frameworks'. The Kotlin Gradle Plugin will use the CONFIGURATION and SDK_NAME environment variables as subdirectories (those variables will be provided by Xcode)

iOS: Creating/Showing the ViewController

Since we have connected Gradle (Kotlin Compile) to our Xcode project and wired everything up, we can implement the UIViewController which can show our Compose UI. Similarly to androidMain, we can write code, specifically for iOS in the source set called iosMain. In there, we can use the ComposeUIViewController function as an entry point into our Compose app.

/* app/src/iosMain/kotlin/.../SampleViewController.kt */
package io.sellmair.app

import androidx.compose.ui.window.ComposeUIViewController

@Suppress("unused") // Used by Swift
fun create() = ComposeUIViewController {
    MainScreen()
}

It is fair to suppress the 'unused code' warning in IntelliJ, as this code might only be used by Swift.

Note: Kotlin Multiplatform Tooling in Fleet can provide Kotlin <-> Swift cross-language capabilities. You're welcome to check this out and provide feedback.

The SampleViewController.create can now be used inside Xcode to present the UI on screen. When using SwiftUI, it is as easy as implementing a UIViewControllerRepresentable and displaying the ComposeView

import SwiftUI
import KmpApp // <- Our Kotlin Framework

struct ComposeView: UIViewControllerRepresentable {
    func makeUIViewController(context: Context) -> some UIViewController {
        return SampleKmpViewControllerKt.create()
    }

    func updateUIViewController(_ uiViewController: UIViewControllerType, context: Context) {

    }
}

struct ContentView: View {
    var body: some View {
        ComposeView()
    }
}

✅ After launching the iOS app and seeing the Compose UI on the screen, we can call the iOS setup to be done

Setting up the Desktop target

Now that the application shows the UI successfully on iOS as well as Android, the remaining target to configure is the "Desktop".

Again, similar to the androidMain and iosMain source sets, the jvmMain source set can be used to declare dependencies, specifically for the JVM, as well as put code for the JVM only.

As of writing this article, "Compose for Desktop" requires adding one compose.desktop library.

kotlin {
   sourceSets.jvmMain.dependencies {
        implementation(compose.desktop.currentOs)
    }
}

After the project synced with Gradle, we can create a Main.kt file under src/jvmMain/kotlin and use the convenient application {} and Window {} functions to show our Compose UI.

package io.sellmair.app

import androidx.compose.ui.window.Window
import androidx.compose.ui.window.application

fun main() = application {
    Window(title = "KMP Demo", onCloseRequest = ::exitApplication) {
        MainScreen()
    }
}

After pressing the green 'run gutter' within the IDE, our Compose Desktop App should show on the screen.

Attribution

Title Wizard Hat Image by Vectorportal.com, CC BY

One of such seemingly universally agreed upon idioms seems to be "Composition over Inheritance" and During my entire career at JetBrains, I never heard any engineer even remotely trying to argue with this. The tradeoffs of what most devs would call 'composition' over 'inheritance' seem to be very well understood, and in most cases it is very much advisable not use abstract classes! Let me even drop the first very controversial statement early on in this post:

If you are a young engineer, you should never use any abstract class unless you are very, very sure of what you are doing. Just do not. Pretend that your favourite language (since you're reading this, Kotlin, I assume), does not have such a language feature.

There are some good exceptions, and I will show two cases where I, personally, like using abstract classes.

Expectation vs Reality

The biggest downsides of abstract classes are

• Bad code re-use-ergonomics: You can only inherit code from one superclass, therefore, code sharing can only be 'linear' which is very limiting, often leading to code duplication or bad abstractions.

• Missing scopes and clear boundaries / tight coupling: Sharing logic with abstract classes mostly leads to complex class hierarchies. Understanding small pieces of code in isolation becomes impossible because every unit of code depends (and potentially interacts) with all other units. To understand one class, you need to understand all classes in the hierarchy. To understand one function, you need to read and understand all private states this function has access to and all other private/protected functions it can call.

• Difficulty in Testing: Since code is tightly coupled and no isolated units of code even exist, it can become quite hard to test in abstract base classes.

• ...

And even though many more problems can be listed against using abstract classes, the reality is (at least for me): Real life code is filled with classes like AbstractFoo or BaseXyz used for sharing code. I would like to complain even further: Most of the time I struggled hard to understand code, when most time was spent not writing code, but wondering 'where to extend or modify existing code', when frustration rose high: It was almost always some abstract class hierarchy, sharing logic.

Refactoring abstract classes: My recipe

Over the time of my career, I have encountered frustrations with such abstractions often enough that I am now following a pretty easy 'refactoring recipe' which I would like to share with practical code examples. Note: As with any other recipe you will find online, this one here is also influenced heavily by my own personal taste. However: As with other recipes: If you don't like it too sweet: Just put in less sugar! If you prefer more spice, use more chilies 🌶️.

Showcase: Kotlin's notion of a 'Compilation'

My previous job at JetBrains was to work on Kotlin Multiplatform, fixing many issues, working on the technology for several years until reaching an initial 'stable release'.

One of the hardest to work with parts in the build tooling code, which still gives me nightmares, was the code we had for modelling all the different kinds of 'compilations' we support in Kotlin.

Here is an absolutely horrendous diagram of this code before the refactoring:

You can see: There is no clear structure (granted, maybe other ways of building diagrams would potentially yield some structure). Working on such class hierarchies is almost impossible. Debugging is a nightmare, fixing bugs takes crazy amounts of time.

By applying the recipe, which is shared in this blogpost, the code was refactored to look something like this:

Step 1: Analyse your class hierarchy

The example shown in the blogpost can be found here

GitHub: Example Code: Inheritance

GitHub: Example Code: Composition (Result)

In order to make this recipe practical, we will use an imaginary code example: In this example, let's pretend to have some entity as a GameState representing the current state of a Game. The abstract class hierarchy we would like to refactor is used for saving this GameState . The abstract class is called AbstractGameStateSaver

/**
 * Base class for saving the current state:
 * See the [save] method accepting a 'GameState' that can be persistet
 */
abstract class AbstractGameStateSaver {
    open fun save(state: GameState) {
        storeGameState(
            encodeGameState(state)
        )
    }

    /*
    To persist the "GameState" we need to encode the state to bytes and then store
    those bytes 'somewhere'
     */
    protected abstract fun encodeGameState(gameState: GameState): ByteArray
    protected abstract fun storeGameState(byteArray: ByteArray)
}

IntelliJ allows us to see a nice diagram of the AbstractGameStateSaver hierarchy. For the sake of simplicity in this example, the hierarchy is kept not too complicated here as well. However, the recipe will work equally well for more complex hierarchies! It's just harder to put more complex cases in the format of a blogpost!

Analysing the hierarchy will provide us with some kind of intuition of what code might be actually shared! It seems, just by looking at it that the game has four implementations of saving the current state. However, there might just be two dimensions:

1. Format: Two formats for storing the Game state are supported: Json and Java Serializable

2. Manual vs Auto save: It seems that the implementations somehow differentiate between saving manually (assumable by a player pressing some kind of save button) and automatic saving

Both dimensions are combined to offer four implementations

• Manual Save, Json

• Auto Save, Json

• Manual Save, Java Serializable

• Auto Save, Java Serializable

Step 2: Take the abstract base class and convert it to a non-abstract class

This means we will convert

abstract class AbstractGameStateSaver { to class GameStateSaver {

We can do this by also replacing all abstract functions with 'initial draft' interfaces that get passed to the GameStateSaver 's constructor. This saver requires some function to encode the current GameState to a ByteArray? Sure, let’s convert the encodeGameState function to an interface. Some implementation to write the encoded ByteArray to some persistent place is required? Sure, let’s create an interface!

// Before
abstract class AbstractGameStateSaver {
    open fun save(state: GameState) {
        storeGameState(encodeGameState(state))
    }

    protected abstract fun encodeGameState(gameState: GameState): ByteArray
    protected abstract fun storeGameState(byteArray: ByteArray)
}

// After
class GameStateSaver(
    private val encoder: Encoder, 
    private val writer: Writer,
){
    interface Encoder {
        fun encodeSaveState(gameState: GameState): ByteArray
    }

    interface Writer {
        fun writeSaveState(encoded: ByteArray)
    }

    fun save(gameState: GameState) {
        gameStateWriter.writeSaveState(
            encoder.encodeSaveState(gameState)
        )
    }
}

Step 3: Implement interfaces for the previously analysed use-cases

By also using composition!

For the sake of simplicity, all implementations only support writing to the Game state to a file (see that every class inherits from AbstractFileGameStateSaver 🎉) It should be simple to convert the code! Let’s look into this abstract class as well!

/**
 * Base Class for 'File based' Save States:
 * Will encode the [GameEntity] parts of the GameState and write it to the provided [file]
 */
abstract class AbstractFileGameStateSaver(private val file: File) : AbstractGameStateSaver() {

    /**
     * Easy: We know how to store the bytes in a file!
     */
    override fun storeGameState(byteArray: ByteArray) {
        file.writeBytes(byteArray)
    }

    /*
    This abstraction relies on somebody else providing the exact implementation
    of encoding the individual game entities
    */
    abstract fun encodeGameEntity(gameEntity: GameEntity): ByteArray

    /*
    This abstraction relies on somebody else being able to create the "SaveFileHeader"
     */
    abstract fun createSaveStateHeader(): SaveStateHeader

    open fun writeGameEntityHeader(
        stream: ByteArrayOutputStream, gameEntity: GameEntity, encoded: ByteArray
    ) {
        stream.write(ByteBuffer.allocate(4).also { buffer -> buffer.putInt(encoded.size) }.array())
    }

    open fun writeGameEntity(stream: ByteArrayOutputStream, gameEntity: GameEntity, encoded: ByteArray) {
        writeGameEntityHeader(stream, gameEntity, encoded)
        stream.write(encoded)
    }

    override fun encodeGameState(gameState: GameState): ByteArray {
        return ByteArrayOutputStream().also { out ->
            val header = createSaveStateHeader()
            writeGameEntity(out, header, encodeGameEntity(header))
            writeGameEntity(out, gameState.levelState, encodeGameEntity(gameState.levelState))
            writeGameEntity(out, gameState.playerState, encodeGameEntity(gameState.playerState))
        }.toByteArray()
    }
}

Oouf 😰: A typical example of an abstract class being an abstract class: A lot is going on and it might not be immediately clear, at first sight, what should be done here? Luckily, the override for storeGameState looks actually simple: The class requests a File in the constructor and will just store the encoded state to this file. The remaining code seems to belong to the Encoder part (for later).

This storeGameState can easily be isolated and extracted into an implementation of the previously established Writer

class FileGameStateWriter(
    private val file: File
) : GameStateSaver.Writer {
    override fun writeSaveState(encoded: ByteArray) {
        file.writeBytes(encoded)
    }
}

First interface implementation done ✅

The encoding part, however, still relies on some abstract methods. We therefore just create a Encoder implementation and will take the same approach: Convert each abstract function to a new interface which will be requested in the Encoder's constructor!

class GameStateEncoder(
    private val saveStateHeaderFactory: SaveStateHeaderFactory,
    private val gameEntityEncoder: GameEntityEncoder
) : GameStateSaver.Encoder {

    /**
     * The encoded Game State will include some kind of header. 
     * This information will include whether or not the save was done automatically
     * and when the save happened.
     */
    interface SaveStateHeaderFactory {
        fun createSaveStateHeader(): SaveStateHeader
    }

    /**
     * The [GameState] consists out of several parts ([GameEntity]). 
     * We will encode each part individually with this encoder
     */
    interface GameEntityEncoder {
        fun encodeGameEntity(gameEntity: GameEntity): ByteArray
        companion object
    }

    override fun encodeSaveState(gameState: GameState): ByteArray {
        return ByteArrayOutputStream().also { out ->
            val header = saveStateHeaderFactory.createSaveStateHeader()
            out.write(gameEntityEncoder.encodeGameEntity(header))
            out.write(gameEntityEncoder.encodeGameEntity(gameState.levelState))
            out.write(gameEntityEncoder.encodeGameEntity(gameState.playerState))
        }.toByteArray()
    }
}

Step 4: Repeat until all interfaces are fully implemented

In the current example, there are two more interfaces to implement: SaveStateHeaderFactory and GameEntityEncoder: For this blogpost I will only show implementing the GameEntityEncoder for json and Java Serializable.

We therefore copy the respective code, again, from the abstract class into an implementation of the previously created GameEntityEncoder.

// Before
abstract class AbstractJsonFileGameStateSaver(file: File) : AbstractFileGameStateSaver(file) {
    override fun writeGameEntityHeader(
        stream: ByteArrayOutputStream, gameEntity: GameEntity, encoded: ByteArray
    ) {
        stream.write("json".encodeToByteArray())
        super.writeGameEntityHeader(stream, gameEntity, encoded)
    }

    override fun encodeGameEntity(gameEntity: GameEntity): ByteArray {
        return gameEntity.encodeAsJson().encodeToByteArray()
    }
}

// After
object JsonGameEntityEncoder : GameStateEncoder.GameEntityEncoder {
    override fun encodeGameEntity(gameEntity: GameEntity): ByteArray {
        // Assume we have the encodeAsJson implemented somewhere
        val jsonAsBytes = gameEntity.encodeAsJson().encodeToByteArray()

        return ByteArrayOutputStream().also { out ->
            out.writeString("json")
            out.writeInt(jsonAsBytes.size)
            out.write(jsonAsBytes)
        }.toByteArray()
    }
}

Note: Just by reading the AbstractJsonFileGameStateSaver it is very hard to understand how the actual encoding would work. This is because of the opaque relation between writeGameEntityHeader and encodeGameEntity . Such hard to reason about code is a widespread symptom of abstract class hierarchies (and was purposely written this way)

Step 5: Refine the new created interfaces and composable implementations

Refine the interfaces and their implementations, provide nice discoverable APIs, and provide factory functions!

For this step, companion objects will be used in Kotlin to make certain APIs discoverable and uppercase factory functions to create actual instances of our GameStateSaver. Again: If you do not like it sweet, feel free to put less sugar into the cake!

Here is an example, making the previously implemented FileGameStateWriter file private and providing some discoverable API by extending the interfaces companion:

fun GameStateSaver.Writer.Companion.file(file: File): GameStateSaver.Writer = FileGameStateWriter(file)

private class FileGameStateWriter(
    private val file: File
) : GameStateSaver.Writer {
    override fun writeSaveState(encoded: ByteArray) {
        file.writeBytes(encoded)
    }
}

Step 6: Declare the final entities

In Step 1, we were able to see that we had four implementations of the GameStateSaver . To finish our refactoring, we need to declare 4 such GameStateSaver as well.

Previously, these four implementations were just this: Implementations of the respective base classes. This was actually quite simple, yet very inflexible:

/**
 * Implementation of a save state which is
 *  - file-based
 *  - uses json as encoding
 *  - creates a header with current time and 'isAutoSave = true'
 */
class AutoSaveJsonFileGameStateSaver(file: File) : AbstractJsonFileGameStateSaver(file) {
    override fun createSaveStateHeader(): SaveStateHeader {
        return SaveStateHeader(isAutoSave = true, saveTime = Clock.System.now())
    }
}

/**
 * Implementation of a save state which is
 *  - file-based
 *  - uses java.io.Serializable as encoding
 *  - creates a header with current time and 'isAutoSave = true'
 */
class AutoSaveJavaSerializableFileGameStateSaver(file: File) : AbstractJavaSerializableFileGameStateSaver(file) {
    override fun createSaveStateHeader(): SaveStateHeader {
        return SaveStateHeader(isAutoSave = true, saveTime = Clock.System.now())
    }
}

/**
 * Implementation of a save state which is
 *  - file-based
 *  - uses json as encoding
 *  - creates a header with current time and 'isAutoSave = false'
 */
class ManualSaveJsonFileGameStateSaver(file: File) : AbstractJsonFileGameStateSaver(file) {
    override fun createSaveStateHeader(): SaveStateHeader {
        return SaveStateHeader(isAutoSave = true, saveTime = Clock.System.now())
    }
}


/**
 * Implementation of a save state which is
 *  - file-based
 *  - uses java.io.Serializable as encoding
 *  - creates a header with current time and 'isAutoSave = false'
 */
class ManualSaveJavaSerializableFileGameStateSaver(file: File) : AbstractJavaSerializableFileGameStateSaver(file) {
    override fun createSaveStateHeader(): SaveStateHeader {
        return SaveStateHeader(isAutoSave = false, saveTime = Clock.System.now())
    }
}

Now we can just compose our instance using our newly introduced building blocks. To do so, the following code will use uppercase factory functions:

fun AutoSaveJsonGameStateSaver(file: File): GameStateSaver {
    return GameStateSaver(
        gameStateWriter = GameStateSaver.Writer.file(file),
        encoder = GameStateEncoder(
            gameEntityEncoder = GameEntityEncoder.javaSerialization,
            saveStateHeaderFactory = SaveStateHeader.factory(
                isAutoSave = true
            ),
        ),
    )
}

fun AutoSaveJavaSerializableGameStateSaver(file: File): GameStateSaver {
    return GameStateSaver(
        gameStateWriter = GameStateSaver.Writer.file(file),
        encoder = GameStateEncoder(
            gameEntityEncoder = GameEntityEncoder.json,
            saveStateHeaderFactory = SaveStateHeader.factory(
                isAutoSave = true
            ),
        ),
    )
}

fun ManualSaveJsonSaveGameStateSaver(file: File): GameStateSaver {
    return GameStateSaver(
        gameStateWriter = GameStateSaver.Writer.file(file),
        encoder = GameStateEncoder(
            gameEntityEncoder = GameEntityEncoder.json,
            saveStateHeaderFactory = SaveStateHeader.factory(
                isAutoSave = false
            ),
        )
    )
}

fun ManualSaveJavaSerializableGameStateSaver(file: File): GameStateSaver {
    return GameStateSaver(
        gameStateWriter = GameStateSaver.Writer.file(file),
        encoder = GameStateEncoder(
            gameEntityEncoder = GameEntityEncoder.json,
            saveStateHeaderFactory = SaveStateHeader.factory(
                isAutoSave = false
            ),
        )
    )
}

At this point, the positive as well as the negative points about this approach should be rather clear

• The composition is explicit and more verbose

• The composition can be easily extended

  Just imagine adding support for writing theGameStateto an http server. A nightmare to extend the abstract classes, just one more implementation of theWriterinterface using the composition

• The composition can re-use components very easily

  NewGameStatewriter instances can be declared freely, re-using any arbitrary building block which currently exists.

• Any source file in the composition can be understood on its own!

If you're a Kotlin application developer, the above code might remind you of writing user interfaces withCompose. While 'Compose the UI framework' leverages many more advanced techniques, the concept of 'composition' is contributing heavily to the ergonomics of Compose and likely fundamental to its success.

Reasons to use abstract classes

There might be actually many good reasons for using abstract classes, I guess? But as promised at the beginning of this blogpost, I now list the two cases where I, personally, use abstract classes:

junit 4 (a.k.a for historical reasons)

Do I need to say much more? My daily job is to contribute to either kotlin.git or intellij.git. Both projects already accumulated a lot of history, and many fundamental tests and test infrastructures rely on junit 4 therefore, naturally, share plenty of testing code using inheritance. This is legacy and might change, maybe, but I actually think that writing tests using inheritance is kind of ok.

interface vs abstract class

Sometimes, rarely, but sometimes, I actually want to enforce that an implementor of a given contract (represented either as interface or abstract class) cannot implement another such contract. In this case, using abstract classes seems like a perfect solution to me:

sealed interface Result
interface Success: Result
interface Failure: Result

class WeirdResult: Success, Failure // 😨

sealed class Result
abstract class Success: Result
abstract class Failure: Result

class SuccessImpl: Success() // Cannot also implement Failure()!
class FailureImpl: Failure() // Cannot also implemente Success()!

Wrap up: Step by Step, The recipe

• Step 1: Analyse your class hierarchy

  Use some code analysis tool of your choice (I recommend IntelliJ) and get some insight into the structure of the abstract class hierarchy:

  • Look for 'all implementations of this given abstract class'

  • Try to find some structure in the hierarchy.

• Step 2: Take the abstract base class and convert it to a non-abstract class

  • Drop the abstract keyword in the class declaration

  • Replace all abstract functions with interfaces

  • Inject implementation of those interfaces in the constructor of the class

• Step 3: Implement interfaces for the previously analysed use-cases

  • Go to direct subclasses of the recently converted abstract class and try to convert the override fun functions into implementations of the new interfaces

  • If the subclass was also declared abstract, then convert those abstract functions into interfaces again

• Step 4: Repeat until all interfaces are fully implemented

  Recursively apply Step 3, work your way up to the actual final implementations, copying code from the override fun functions into implementations of composable interfaces.

• Step 5: Refine the new created interfaces and composable implementations

  Stir, Stir Stir 🧑‍🍳, the recently created interfaces and implementations need some refinements. Find good names, find good abstractions, put some love into the recently refactored code!

• Step 6: Declare the final entities

  Provide factory functions to allow easy constructions of the 'production' instances. The code should feel declarative and 'composable'.