Notes on seeking wisdom and crafting software

Platforms are inevitable

A platform is an enabler abstraction in the technology stack. It hides substantial complexity with a simple usage contract for the layers above. Similarly, the developer experience provided by a platform invites the creators to start building on top. We had done a tour of working in a platform earlier. I recently learned a thing or two on how platforms come into being. Let’s dive in.

Organic growth and refactoring

In the beginning there is just one app. Business problem is paramount, and it uses a tech stack to serve the customer needs. Over time the app becomes successful. Business scales more and now needs more apps sharing the same functionality and thus replicate the success.

Each app could build their own thing 1. It’s faster and much too gratifying. The high is short-lived since the lessons have to be relearned. Remember maintenance, hot-fixes and their allies? At one point of time, the company decides to do a DRY removal.

A platform originates when we refactor the duplicate functionality in the organization and strategically align the group to stay closest to the business goal.

While refactoring, the order of priority is usually: alignment to business, alignment of people solving the problem and then the architecture 2. Because of the first two, the architecture may evolve into a separate service providing this capability to more than one app.

Now we have one group solving one critical aspect of the business problem. This group can bring in innovations that just multiply. Every app wins from this investment.

Deep pockets of problems

Sometimes platforms are a pre-meditated option. Deep problems are usually a platform concern.

  1. These could be deep computer science problems. E.g., key-value stores, a language model service making the GPU caching and inference efficient etc.
  2. These could be deep domain problems. E.g., payments along with the complexity of dealing with multiple providers, or geographical regulations.
  3. They can be infrastructural problems. E.g., making operations efficient with fast cloud deployments, deterministic build and deployment pipelines, developer experience etc.

These patterns are well-known in the industry, and it is a matter of time when a company reaches a stage to start investing on these pockets.

Refactor the dependency and build your own

The third origin of platform is a middle path between both of the above.

In the beginning we outsource capabilities to startup-friendly service providers. E.g., Stripe for payments, or XYZ for CI/CD pipelines. When the company grows sufficiently large, they want to start building the service in-house to optimize on overall cost.

A platform is created to replace the external dependency.

Shipping the organization structure

We are positing that evolution of platforms in innate in the process of scaling an app (and the related business). There maybe a few side effects of this evolution.

  1. Sum of parts may be worse than the whole. Few bugs may demand tremendous effort across the platform and their downstream or upstream dependencies.
  2. Corollary to above: metrics at each org can be green, but the overall result can be red. E.g., for Payment (think Stripe here), the metric need not be overall conversion of the flow. They’ll only care of reliability of that step! However, because a provider is missing, the overall flow can suffer (how do you measure reliability of something that doesn’t exist :P).
  3. The only solution I could see is to drive an end to end OKR at the common parent in the org chart. This is no longer a technical problem.

Conclusion

Three reasons to create a platform: (a) reuse and focus on innovation to provide value to the downstream apps, (b) deep problems and well known patterns, and (c) replace a dependency with in-house platform to optimize cost or efficiency.

Every app grows enough over time to be refactored into a platform and the rest. Platform enables the rest of the team to focus purely on the business value.

Beware! Sum of the parts may be worse than the whole. It requires alignment between the platform, its downstream and upstream dependencies to drive features and fixes.

Will you start with a platform from day one? No, may be not. Stay monolithic until you’ve data to prove you need a refactoring. Well, from day one, do ensure that refactoring is not too costly. There must be an equivalent of writing unit tests for the org 😉.

Footnotes

  1. Duplication is real when the organizations building both apps are sufficiently far from each other; even within the same company. Sometimes we give speed as a reason, other cases it is just a variation of the not invented here syndrome.

  2. After a few years, we usually retrospect to say that the organization structure is shipped to the customer. Myopic evolution inevitable if we build a platform for two apps. The solution is to decouple and build for a wide consumer set. Embrace diversity by design wherever possible.