Concatenability principle
Today I’d like to talk about the principle of concatenability in software architecture. I’m not sure if this is the right word, but I could not find any better term for that.
The general idea behind that could probably be easily recognized: modularity is quite close to that, but still different, and so is composability.
I do not have a precise definition right away: this is actually one of the goals that I’m trying to achieve with this substack. But I see that thing everywhere nowadays and so I will provide lots of examples.
Concatenability manifests in two phases: adding and removing.
Concatenability: an example
Let’s take any sufficiently big website, such as Twitter. Let’s look at one of its smaller features, for example the “Add to bookmarks” button below each tweet. You can bookmark the tweets that you expect to be useful later; you can find the bookmarked tweets on a special “Bookmarks” page.
Now imagine that Twitter must now remove this feature due to some mysterious legal-ish reasons. Nobody knows why, but a) there must be no trace of that this feature anywhere on the site; b) the codebase must be fully cleaned up: you cannot just comment out a couple of buttons in HTML and call it a day; c) of course, the data about the bookmarked tweets also must go; d) the sanctions for not doing it are so vaguely serious that it needs to be done, period. The deadline is quite comfortable, say a couple of months. You don’t need to rewrite the Git history: the clean trunk would be enough.
What do we need to do? Something like this comes to mind:
remove the button from HTML, remove the SVG icon;
remove the links to the “Bookmarks” page, including icons;
remove the code responsible for handling the “Bookmarks” page, make the page return 404 using the normal unknown page handler;
remove the template for this page and all the CSS classes;
drop the table where the bookmarks data is stored, drop various secondary data;
change various machine learning models so that they no longer use the bookmarks data; retrain models;
remove the stats pages, metrics, alerts etc.
remove the tests from our test suite;
remove the code and assets from both iOS and Android clients;
remove mentions of bookmarks from API documentation;
remove mentions of bookmarks from help pages;
maybe some even smaller things would need to be removed.
While we’re doing all that, we continue our normal deployments routine, of course. Also, other engineers continue their usual development, except that they can no longer implement features that depend on bookmarks.
Two months later we’re done. We had to remove many different things on almost all layers of our software system: HTML, JavaScript, the backend language that we use, CSS, plain English text, database tables, Python data processing scripts, Graphite expressions, Swift and Kotlin, etc., etc.
Also, we’ve learned quite a bit about our system along the way. Some pieces of our system were pretty modular: we could remove just a single CSS file “_bookmarks.css”, and, say, a couple of backend modules that clearly have “bookmarks” somewhere in the name. Then we had to remove mentions of those modules, and our software compiled and worked. Some parts of the system were harder to untangle, for all kinds of reasons.
We can now step back and get a bird’s-eye view of all the code that we had to touch: the code that used to comprise the “Bookmarks as a feature”. This code is cross-technology (full-stack), so we cannot call it “Bookmarks module”, because a module is usually something that lives only in a single technology, such as CSS or Java. We have to find a word for that.
What happened in our codebase is one of the phases of concatenability: removing.
Concatenability: adding
Few months later the Mysterious Council appears in the dreams of all vice-presidents simultaneously, apologizes and explains, among other things, that for still vague reasons the concept of a bookmark is no longer forbidden and the company can re-add it to the product if still necessary. Well, bookmarking is a pretty useful feature, so let’s add it to our backlog.
We now have a chance to implement bookmarking in a different way, maybe somewhat more modular. We understand the feature very well, and we better understand the codebase from our previous experience of removing this feature.
Basically, this time the code will be more concentrated, and less spread out in unrelated files. We can’t help but notice that the code will be easier to remove again, if need be (we can no longer discount the chance that the Mysterious Council will change its mind again).
This is another phase of concatenability: adding.
Database modeling and concatenability
It’s always easier to add something to the typical software system than to remove it. But what if from the start we could maintain awareness that almost everything will eventually need to be removed, and plan accordingly?
So, in the future posts we’re going to evaluate every possible database design decision taking into account the idea of concatenability. We’ll see that it manifests on every layer: logical database model, physical database model, data itself, secondary data processing, data consumers, database storage technology, server implementations etc.
In the previous post we discussed representing either/or data in JSON representation. This is a small, but very good example that would be helpful to illustrate concatenability. Let’s discuss it next week.