How We Built Whimsical

It's All About Balance
Kaspars Dancis Posted on Feb 23, 2018 in Tech

I started Whimsical in the summer of 2017. The plan was to build a tool that would let people collaborate on ideas visually in a way that’s simple and enjoyable. A bit like Dropbox Paper but you get flowcharts and wireframes instead of plain text.

Visual collaboration is not an entirely novel idea, of course, but the existing tools were all cumbersome and uninspiring. So I set out to build the opposite – fast, simple and with obsessive focus on thoughtful design.

It also made a lot of sense to apply the same ethos to our tech stack:

Fast. You want to move as fast as possible with a small team. Speed is the ultimate startup advantage, and vice versa, being slower than other companies is the surest way to go out of business.

Simple. Tools that have less features are easier to understand fully and are less likely to break. In contrast, complex tools will often be half-understood. Which in turn will lead to mistakes and surprising issues. Which in turn will slow you down.

Thoughtful design. Poorly designed tools may seem to do the job at first but will gradually lead you deeper and deeper into a swamp that will be very costly to escape. Which in turn will slow you down.

The Architecture

Open in Whimsical

Whimsical is a so called single-page web app (SPA). Most of the logic is implemented in the client-side application (“app”) that runs in the browser and syncs updates to the backend (“cloud”) via RPC-style HTTP requests. “Cloud” notifies “app” about updates made by other users via WebSocket.

All of the application’s data are stored in a PostgreSQL database. This gives us the comfort of ACID properties, while leaving a lot of room to scale, especially now that AWS provides a PostgresSQL-compatible Aurora variant. We also use Redis for publishing and subscribing to events, as well as storage of transient data such as who is currently viewing a particular doc.

Finally there’s a tiny service (“imager”) we built for generating thumbnails via a headless Chrome.

All of the services and data stores run in Amazon’s WebServices. On the public Internet side, we have an Application Load Balancer (ALB) that distributes all requests to whimsical.co to a cluster of NGINX instances (“edge”). From there, “edge” routes the requests to either “cloud” or Amazon’s S3 for static resources, such as website or images. Since there are multiple instances of “cloud” running, it also has an internal load balancer in front of it.

Clojure and ClojureScript

One of the most far reaching tech decisions a startup needs to make is what programming language(s) to use. For example, Facebook to this day runs mostly on Hack, a language that is derived from PHP that Facebook got started with. The switching cost was so high that they ended up forking the language.

For Whimsical, we chose Clojure and its sibling ClojureScript. I discovered Clojure a few years ago as a saner alternative to Java for building backend services running on JVM. Later on, we successfully moved a fairly sizable front-end app, as well as parts of native mobile apps, to ClojureScript.

Having one language across the whole stack is a major speed boost, as it significantly reduces mental overhead of switching between parts of the stack. To make things even sweeter, you can reuse code across the whole stack, especially when you have three different frontends – web, iPhone, and Android.

The ultimate power of Clojure however lies in the speed. Everything in Clojure is designed for rapid development. The lightweight syntax. REPL. Pure data. This is why I firmly believe it’s the best programming language for startups.

The one criticism that Clojure typically gets is the lack of static typing. There are times when I miss it too. But for early stage startups where you do a lot of experimentation, have a small team, and want to move as fast as possible, the overhead of static types will only slow you down.

Web Frontend

Facebook might be stuck with PHP, but they also brought us React, which has had a major impact on how non-trivial web apps get built. What’s particularly interesting is that it turned out to be a perfect match for the functional programming paradigms available in Clojure.

For Whimsical, we picked Reagent, an open source library for building React-based apps in ClojureScript. What’s really powerful about Reagent is that it takes the concept of the React component and strips it down to the essence – a simple function that takes data as input and produces UI as output.

Reagent, like React, is a small library focused on doing just one thing – rendering UI. It doesn’t dictate how the application state gets managed and synchronized with the backend. For bigger applications, it’s a good idea to adopt a unified pattern for state management. That’s where Re-frame, Reagent’s best buddy, comes in. It gives us an elegant scalable pattern for managing the application’s state in a purely functional manner. Besides, its README is pure gold.

Another area where ClojureScript is pushing the state of the art is the development workflow, mainly thanks to Figwheel. It’s a little gem that gives you a REPL connected to a browser session, as well as automatically pushing code changes to the client. The code push is especially magical because it can update the running app in place without having to reload the browser. This way, the whole application state is preserved and it’s also much faster. And did I mention it also works with native apps?

Backend

The backend service for Whimsical mostly serves as a lightweight layer between the client and the data store. There is of course some critical logic but other than password hashing, none of it is CPU intensive. It made a lot sense to make it asynchronous.

Unsurprisingly, Clojure is perfect for this. We use Aleph as an HTTP/WebSocket server and for the application logic rely on the core.async library and its friend full.async. core.async is another great example of Clojure’s power as it enables to write asynchronous code without callbacks. It’s a bit like await/async in JavaScript - but it’s just a library rather than core language construct.

Managing Infrastructure

Managing infrastructure can be messy and time consuming. One solution to this is using a higher level service such as Heroku to manage the infrastructure for you. We considered this at first but ultimately couldn’t justify the extra ongoing costs and constraints.

Instead, we decided to invest a bit of time upfront and implement a toolchain for automating the infrastructure management as much as possible. It wasn’t entirely painless but now that we have it in place, it takes the same effort to deploy things and keep them running as with Heroku but we have gained way more flexibility and visibility into the runtime environment.

Every service gets packaged as a Docker container and is stored in a centralized registry. We tag each version with a git commit hash and also store the currently deployed versions in git (in a monorepo that also contains all the code). This gives us a clear understanding of what the infrastructure looked like at any point in time. The Docker containers also serve a secondary purpose of providing a development environment which closely mimics production.

For managing the actual AWS infrastructure, such as load balancers, security groups, ECS tasks, and so on, we rely on Terraform, which lets us define infrastructure as code in a declarative manner. It’s a bit cumbersome to set up at first but once in place, it’s a huge time saver. To get started, we used Segment’s Stack as a template and incrementally adapted it to our needs. Needless to say, having the infrastructure described in code lets us store it in git with all the associated benefits.

It’s All About Balance

I started practicing yoga recently (it’s awesome). The surprising thing I discovered was the focus on balance and how much positive impact that had on me.

And it dawned on me that balance is the key to building anything great and successful.

When building a software startup, the trickiest thing is balancing between moving fast today and being able to move fast in the future. That’s the one thing I always keep top of mind. That’s the one thing that influences every decision.

Discuss on Hacker News

Collaborate visually. Lightning fast. Find out more