Introducing Crank

By Brian Kim – April 15, 2020

A deep dive into why Crank.js was created, exploring the limitations of React's Suspense API and the philosophy behind using generators and async functions for components.

After months of development, I’m happy to introduce Crank.js, a new framework for creating JSX-driven components with functions, promises and generators. And I know what you’re thinking: oh no, not another web framework. There are already so many of them out there and each carries a non-negligible cost in terms of learning it and building an ecosystem to surround it, so it makes sense that you would reject newcomers if only to avoid the deep sense of exhaustion which has come to be known amongst front-end developers as “JavaScript fatigue.” Therefore, this post is both an introduction to Crank as well as an apology: I’m sorry for creating yet another framework, and I hope that by explaining the circumstances which led me to do so, you will forgive me.

I will be honest. Before embarking on this project, I never considered myself capable of making a “web framework.” I don’t maintain any popular open-source libraries, and most of the early commits to this project had messages like “I can’t even believe I’m actually considering making my own web framework.” Before working on Crank, my framework of choice was React, and I had used it dutifully for almost every project within my control since the React.createClass days. And as React evolved, I must admit, I was intrigued and excited with the announcement of each new code-named feature like “Fibers,” “Hooks” and “Suspense.” I sincerely felt that React would continue to be relevant well into the 2020s.

However, over time, I grew increasingly alienated by what I perceived to be the more general direction of React, which was to reframe it as a “UI runtime.” Each new API felt exciting, but I disliked how opaque and error-prone the concrete code written with these APIs seemed. I was unhappy, for instance, with the strangeness and pitfalls of the new Hooks API, and I worried about the constant warnings the React team gave about how code which worked today would break once something called “Concurrent Mode” landed. I already have a UI runtime, I began to grumble whenever I read the latest on React, it’s called JavaScript.

Towards the end, I felt marooned, because on the one hand I didn’t feel comfortable using React anymore, but on the other, I didn’t want to use any of the alternatives either. I agreed with the criticisms which Vue and Svelte advocates lobbed in the direction of React, but I was unwilling to convert to these frameworks because they prominently featured HTML template languages as the main way to use them.

I like JSX. I like the small surface area it provides compared to template languages, which provide their own syntax to do basic things like iterating over an array or conditionally rendering something. Meanwhile, the other frameworks which used JSX like Preact and Inferno seemed to follow React blindly in its heroic evolution from “a view layer” into “a UI runtime.” Rather than thinking critically about each new feature, these libraries seemed eager to mimic them for purposes of compatibility, opting to distinguish themselves instead in terms of library metrics like bundle size (Preact) or runtime performance (Inferno). My problems with React weren’t related to bundle size or runtime performance. It was the API itself that needed fixing. I felt like React, which had up to this point been the standard-bearer of JSX, was no longer up to the task of defending its colors.

Tired of the Suspense

The tipping point for me was React’s perennially unready Suspense API, React’s solution for async rendering. For the most part, I ignored talks and articles describing Suspense, partially because the React team kept signaling that the API was in flux, but mainly because most discussions of Suspense just seemed to go over my head. I assumed they would work it out, and we’d eventually have something like async/await for React components, so I continued to incorporate React into my projects without thinking too hard about the future of React and promises.

This was until I decided to explore the Suspense API for myself, when I was trying to create a React hook for usage with async iterators. I had created an async iterator library that I was proud of (Repeater.js), and I wanted to figure out a way to increase adoption, not just of the library, but also of async iterators in general. The answer seemed logical: create a React hook! At the time, it seemed like every API in existence was being transformed into a React hook somehow, and I thought it would be nice for there to be hooks which allowed developers to use async iterators within React components as well.

The result of this effort is available on GitHub, and the library is usable, but I mostly abandoned the effort and any sort of greenfield React development when I came to understand what Suspense was and how unwieldy it would have been to incorporate Suspense into the hooks I had written. As of April 2020, the mechanism behind Suspense is for components which make async calls to throw a promise while rendering to indicate that the component is doing something asynchronously. “Throw” as in the way you would throw an error in JavaScript with the throw operator. In short, React will attempt to render your components, and if a thenable is thrown in the course of rendering, React will catch it in a special parent component called Suspense, render a fallback if sufficient time has elapsed, and when the promise has fulfilled, attempt to render the component again. I say “as of April 2020,” because the React team has consistently said the exact details of the Suspense API might change and has used this declaration to preempt any possible criticisms of this mechanism. However, as far as I can tell, that’s how it will work, and how everyone who has written libraries featuring Suspense assumes it will work.

If this mechanism sounds wild to you, that’s because it is. It’s an unusual way to use promises and throw statements in JavaScript. And I could almost get past this, trusting that the React team knew what they were doing, until I understood the add-on ramifications of this design decision. When a component throws a promise to suspend, most likely that component has not rendered, so there’s no state or refs or component instance which corresponds to this thrown promise. And when the thrown promise fulfills, React will attempt to render the component again, and hopefully whatever API you called which initially threw the promise, an API which would otherwise be ill-behaved in regular JavaScript, would in this second rendering of the component, not throw a promise but return with the fulfilled value synchronously. This means that it doesn’t even matter what the thrown promise fulfills to; instead, it’s an elaborate way to notify React that your components are ready to try and render again.

All of a sudden, what little I had heard about React Suspense made sense. I understood, for instance, why discussions of Suspense almost always involved mentions of a cache. The cache is necessary because there is no component instance on which to store the thrown promise, so when the component attempts to render a second time, it needs to make the same calls to whatever API threw and hope that a promise is not thrown again. And while caching async calls is a useful technique for creating responsive, performant, offline-ready applications, I balked at the idea of this hard requirement of a cache when using promises.

This is because to cache an async call, you need two things. Firstly, you need to be able to uniquely key each call somehow. This is what would allow you to call a promise-throwing function a second time and have it “remember” not to throw a promise again. Secondly, you need to know when to invalidate the cached result. In other words, you need to be able to identify when the underlying data which the cached result represents might have changed, so that you don’t end up showing the user stale data.

Take a step back. Take a high-level look at any application you’re working on. If you’re using promises and async/await, think of the async calls you make, and whether you can both uniquely key each call, and know when to invalidate their results. These are hard problems; in fact, cache invalidation is one of the problems we joke about as being “the two hardest problems in computer science.” Even if you like the idea of caching your async functions, do you want to add this requirement when you’re making a one-off call to some random API, or when you’re trying to bootstrap a demo?

At this point my curiosity sublimated to frustration: Why can’t rendering just be async? Why can’t React components simply return a promise? I scoured GitHub for issues where people suggested this API change, and there was at least one such issue in each of the major JSX libraries (React, Preact, Inferno), but the maintainers either dismissed the issue or did not seem to consider it a high priority. For React, the issue was closed with a comment saying that Suspense would solve everything.

But Suspense solves this problem at the cost of requiring a cache, which as I described feels like such a huge ask. When I went and revisited the actual introductions to Suspense I felt like I was being gaslit. “Suspense allows you to access async data from a server as easily as sync data from memory,” a React maintainer would say in a talk introducing Suspense. But we already have a way to access async data as easily as sync data: it’s async/await syntax, and JavaScript will literally suspend your functions when promises are awaited. The literature on Suspense seemed to invent new problems with promises, like the idea that async code “waterfalls,” which in short just means that code which could run in parallel runs in sequence instead. It’s not a problem, I thought, because we have ways to make async functions run concurrently, for instance, by calling Promise.all() over an array of promises. To me, nothing about React or virtual DOM implementations indicated that we couldn’t use similar solutions, and absolutely nothing indicated that the solution was to throw a promise.

React’s Dogmatic Assertion

I realized Suspense, and the mechanism behind it, was less created because it was the most ideal API; rather, it was borne of a single dogmatic assertion which the React team held and continues to hold, that “[rendering] should be pure, meaning that it does not modify component state, it returns the same result each time it’s invoked, and it does not directly interact with the browser”. Async functions, which are really just functions which return promises, were excluded by definition. Why? Because promises are stateful and therefore “impure.”

Knowing that this was the one immovable axiom from which the React team refused to budge, each of React’s latest design decisions seemed to fall in place. The Suspense and Fiber projects were ways to get around the fact that sync functions could not suspend, and Hooks, React’s much-discussed solution for avoiding class-based components, were really just technical aerobatics to frontload code before return statements.

Correspondingly, a lot React’s pain points began to make sense as well. All of the struggles which React developers faced, like the double-rendering or tree-walking hacks used to hydrate components with async dependencies on the server, or the whole period of collective insanity when React developers thought “render props” were good APIs, could be explained by React’s original sin of requiring rendering to be modeled exclusively as pure functions. The principle leaked into the ecosystem, radiating into developer’s lives by complicating their codebases and architectures when using React.

Freed of this dogmatic assertion, I pondered for a week or so on the kind of JSX-based library you could create if components didn’t have to be sync functions. After all, JavaScript has at present four separate function syntaxes (function, async function, function *, and async function *); wouldn’t it be nice if we could use this entire palette to write components? Could there be a use-case for generator functions as well? Again, the React maintainers dismissed generators by definition, because generator functions returned generator objects, which are stateful and therefore “impure.”

JavaScript is already a UI runtime

At this point, I was intrigued by this idea but I also didn’t want to write a React alternative. I wanted to write applications, not build and maintain a framework. And so I was about to move on to something else, when my previous work with async iterators and generators gave me a flash of insight. The entire React lifecycle, all of the componentDidWhat methods, everything which React was trying to do with classes and hooks and state and refs, all of it could be expressed within a single async generator function.

async function *MyComponent(props) {
  let state = componentWillMount(props);
  let ref = yield <MyElement />;
  state = componentDidMount(props, state, ref);
  try {
    for await (const nextProps of updates()) {
      if (shouldComponentUpdate(props, nextProps, state)) {
        state = componentWillUpdate(props, nextProps, state);
        ref = yield <MyElement />;
        state = componentDidUpdate(props, nextProps, state, ref);
      }

      props = nextProps;
    }
  } catch (err) {
    return componentDidCatch(err);
  } finally {
    componentWillUnmount(ref);
  }
}

This is some pseudo-code I sketched out, where the calls to componentDidWhat() functions merely demonstrate where code goes compared to the React lifecycle. While the actual Crank API turned out to be slightly different, in the moment I felt like I had captured lightning in a bottle. By yielding JSX elements rather than returning them, you could have code which ran before or after the component rendered, emulating the componentWillUpdate() or componentDidUpdate() lifecycle methods. New props could be passed in by stepping through a framework-provided async iterator, which resumed with fresh props whenever the component was rerendered. And the concept of local state, which in React requires calls to this.setState() or the useState() hook, could simply be expressed with local variables, because yielding is not final and the generator’s local scope could be preserved between renders.

Furthermore, you could implement something like the componentDidCatch() and componentWillUnmount() lifecycle methods directly within the async generator, by wrapping the yield operator in a try/catch/finally block. And the framework could, upon producing DOM nodes, pass these nodes back into the generator, so you could do direct DOM manipulations without React’s notion of “refs.” All these things which React required separate methods or hooks to accomplish could be done within async generator functions with just the control-flow operators that JavaScript provides, and all within the same scope.

This idea didn’t come all at once, but it dazzled me nonetheless, and for the first time I felt like the task of creating a framework was achievable. I didn’t know all the details behind how to implement the API above, and I still didn’t know how the framework would handle async functions or sync generator functions, but I saw the start and the end, something to motivate me when I got stuck. And the best part was that it felt like “innovation arbitrage,” where, while the React team spent its considerable engineering talent on creating a “UI runtime,” I could just delegate the hard stuff to JavaScript. I didn’t need to flatten call stacks into a “fiber” data structure so that computations could be arbitrarily paused and resumed; rather, I could just let the await and yield operators do the suspending and resuming for me. And I didn’t need to create a scheduler like the React team was doing; rather, I could use promises and the microtask queue to coordinate asynchrony between components. For all the hard things that the React team was doing, a solution seemed latent within the JavaScript runtime. I just had to apply it.

Not Just Another Web Framework

Crank is the result of a months-long investigation into the viability of this idea, that JSX-based components could be written not just with sync functions, but also with async functions, and with sync and async generator functions. Much of this time was spent refining the design of the API, figuring out what to do, for instance, when an async component is still pending but rerendered, and how things like event handling should work. As it turns out, the simplicity of directly awaiting promises within your components is unmatched by any API the React team has put out, and sync generator functions turned out to be just as if not more useful than async generator functions. I’m very pleased with the result. I literally started tearing up while implementing TodoMVC in Crank, partly because it was the culmination of months of work, but also because it felt so natural and easy.

In 2019 and beyond, there’s been a big push to figure out “reactivity” in each of the web frameworks, how best to track changes to application state and update the UI in response to these changes. What felt like a solved problem became again unsolved, as the various frameworks attempted to refactor their APIs away from classes, so that you could group code by concern rather than when it needed to run in the lifecycle of a component. React invested in “hooks,” and the creation of a custom UI runtime, Vue invested in a Proxy-based observation system, and Svelte invested in a compiler which transpiled variable assignments. On the other hand, Crank uses async and generator functions, language features which have been available in JavaScript since 2015 and are now heavily entrenched in the ecosystem.

By combining these relatively old, almost boring technologies with JSX syntax, I think I’ve created a new way to write components, a way which is more expressive, easier to reason about, and more composable than any of the solutions the other frameworks have provided. But perhaps it’s wrong to say I “created” it. In a sense, Crank is not “just another web framework,” but a design pattern which would eventually have been discovered by the JavaScript community anyways.

And again, I sincerely apologize for creating yet another framework in an already crowded space, but I hope, if you’ve read this far, you understand why I did so, namely, because I thought React was dropping the ball in terms of its newest APIs, because I still wanted to use JSX, and because of the sudden realization that we could be doing so much more with the different function syntaxes available to us in JavaScript.

If any of this interests you, if you want to continue to use JSX over template languages, if you’re tired of debugging hooks, if you want to use promises in your components today, if you’re looking for a framework which has, arguably, the most “just JavaScript” story for reactivity, I encourage you to check out Crank. You can read the documentation or check out the TodoMVC that made me cry a little. Crank is still in its early days, and there’s a lot of work to be done before it can be considered a full-fledged framework, but I think the ideas behind it are sound and I’ve thoroughly enjoyed designing it. I can’t wait to see what people build with it.