I recently spent several weeks refactoring a frontend platform. This post collates some learnings of an ancient developer who doesn’t know a thing about frontend development. Phew, with that disclaimer, let’s start.

Yet another platform (?)

If you’re one of the two readers of this blog, you already know my unprecedented love for platforms. But this time, the problem was on the periphery - the developer experience (DevX). You probably remember that I had promised to never do DevX 7 years back. Anyhow, a desire to learn platform internals got me into the rabbit hole.

We’ve a platform that distributes shared components.

[Marker for awkward pause…]

Wait, did you just solve NPM for the N^th time? No. It distributes binary shared components like those standalone statically compiled C++ libraries. Aha! I knew, you solved CDNs for the zillioneth time in the history of humankind. Nope, more than that. We solved a MxN problem.

• M shared components
• N apps must consume them
• An app can customize the component’s behavior, e.g., inject custom tokens, or redirect to different pages, provide configuration params at runtime, and so on.

We are the glue ¹ that provides a common abstraction to build those M components which will magically work across N apps.

How does the glue work?

A mental model

We have two parts here.

1. M shared components are built using an SDK, fondly termed as “runtime”.
2. N apps use a different SDK, called “loader”.

Loader knows how to find the components by name. It dynamically downloads the component bundle, injects it into the app DOM and calls ReactDOM.render for the component.

There is a shared contract between Loader and the components built with Runtime. Loader always knows what method to invoke after injecting the component bundle.

Where’s the complexity?

1. Architectural boundaries or bounded contexts. What goes inside Runtime?
2. Managing versions of Loader, Runtime and components. What will be the compatibility guarantees?
3. Developer experience: build, debug, test and deploy components. All of these must be self-serve and enablers.

How did you solve these?

1. We created a strict model of what goes “in” a package. E.g., Runtime will be bounded to the core bootstrap and extensibility primitives. NO random utilities like creating an uuid or whatever 🙊.
2. Simplify the compatibility direction. E.g., newer components will work with loaders until X versions. More on this later.
3. We prototyped a create-xyz app like the create-react-app. Scaffolded component ships with Storybook for rapid prototyping. We provide a service to manage latest version and CDN urls for the deployed components.

I will probably do another post about detailed architecture. We’ll stick to the lessons in this one.

Lessons

Source or bundle

Will you ship a NPM package as source files (ts or js) and let the consumer transpile, polyfill and bundle? Or, will you ship as a single binary JS file with all dependencies statically bundled?

Please prefer shipping source files. It allows the downstream apps to control various aspects like tree-shaking, ensuring single copy of dependency and so on.

There are libraries (like react) which do not have any declared dependency and ship as production bundles that just work. Choose this model if you want control on the runtime aspects and your bundle is already pre-optimized, there is not much the consumers can do any ways.

Read you don’t need a bundler for your NPM library↗ for a discussion.

However, shipping as source will expose internals? How do I control the API surface?

Use exports and typings in package.json

NodeJS exports↗ allows specifying various entrypoints to a package. This is supported by various tooling like bundlers or compilers.

Typescript packages can use types and typesVersions fields to specify conditional types. See typescript docs↗ for more details.

With a combination of the above we can restrict the package’s public contract, manage APIs for versions etc.

How will you manage the layering and architectural boundaries within a package?

Static analysis for layering

Layering and a clear demarcation of responsibilities is key to a clean architecture. We can enforce some of these principles with eslint plugins:

1. Use no-restricted-syntax↗ to disable wildcard exports with eslint-plugin-import↗.
2. Use eslint-plugin-boundaries↗ to define various layers and cross component dependencies along with direction of dependency.

During the refactoring, we first authored an eslint configuration file with desired layering. Next, the job was easy, fix the layering issues reported by the linter.

How do you migrate the existing downstream projects to refactored code?

Carve a public API, and move everything else to private

The idea here is similar to that of characterization tests while refactoring legacy code. First, write tests for the legacy code. Then refactor a small bounded context while ensuring that characterization tests are green. Finally, ensure that the newly carved component is adequately tested.

We kept the legacy package as it is. Slowly carved a minimal core Runtime package while ensuring the legacy package continues to work. Certain aspects of the new Runtime package were explicitly exported via /private path in exports and re-exported in the legacy package.

A final cleanup can remove the redirection from legacy to the new Runtime package.

Dependencies dev and peers

This one is the most obvious thing a bunch of oldies like me miss, primarily because of minds polluted by the world of enterprise languages (Java, .NET and alike).

NodeJS exposes 4 kinds of dependencies.

• dependencies are the private upstream packages a package uses.
• devDependencies are the build time dependencies.
• peerDependencies are the packages required at runtime but can be provided by the downstream consumer.
• optionalDependencies and bundleDependencies provide additional metadata.

Do not use dependencies and peerDependencies for the same upstream dependency. dependency will win and peerDependencies has no meaning.

Hoisting and dependency resolution

Another favorite learning for me.

Package managers in the NodeJS world need to ensure that the entire dependency chain (transitive closure of N level of dependencies) must be resolved with each npm install like command. Now, there could be N copies of a dependency. Each package manager provides mechanisms to hoist the dependency by downloading a single copy and then symlinking it to other packages.

Say you have an app with react and fluentui based component. Here’s a common pitfall you may see in monorepos:

Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons: 1. You might have mismatching versions of React and the renderer (such as React DOM) 2. You might be breaking the Rules of Hooks 3. You might have more than one copy of React in the same app See https://reactjs.org/link/invalid-hook-call↗ for tips about how to debug and fix this problem.

What happens is quite interesting. Imagine a monorepo structure like below:

/repo
  /node_modules
  /packages
    /a
      /node_modules
    /b
      /node_modules

• Say we’ve a package in the monorepo using react v18. npm will hoist this package to the /repo/node_modules at root of the repo.
• We add a dependency X which depends on react v17. npm will determine this is a local dependency and cannot be hoisted. Hence it will be downloaded to /repo/packages/a/node_modules
• Now we run the bundler like webpack. Default module resolution will start from the nearest node_modules. It will find react v17 and will bundle it. Further remember the app itself is using react v18 which will be resolved from the parent node_modules.

End result: we have a bundle with two versions of react.

Note that there are other ways this can manifest. E.g., if you’re using npm link then the dependencies are picked from the target of the symlink.

The fix is to tell npm that we need to override the v17 with v18. Fortunately, it is supported in recent versions with an override field in the package.json. See here↗.

I also learned that if you’re part of a monorepo’s core engineering team, managing N versions is going to be your dayjob. It is fugly. More first hand experience here↗.

What if I bite the bullet and let each monorepo package manage its dependencies without hoisting? You can use install-strategy↗ but it doesn’t work with workspaces↗.

You have to look for better package managers. Or build wrappers like https://github.com/microsoft/rushstack↗. Good luck!

That’s all for this learning note. Thanks for reading this far.